[gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 3305 bytes --]

Hi together,

since i am not able to get any real argument or even discussion on IRC nor on this mailing list from
Arfrever (main person behind those changes), i would like to raise the following points now on this
mailing list as told on IRC, so he gets the chance to answer those points and to clear the issues:

-major changes to python eclasses have been done without peer review on this mailing list. This
includes pulling in python-3* versions, even when they are not required nor used on the user system

Our policy is, that major changes to main eclasses should previously shown and discussed publically
on this mailing list, this did not happen for the python eclass. I think, he was already told about
it, but i still did not see any RFC about those eclass changes.
Additionally, those changes now pull in python-3 for every user, also no package does require
python-3 nor will it be used, since the main python version still has to be python-2. This results
in vasted time for additional compilation (both for python-3 and every python module, which is able
to work with python-3) and vasted space on the user system, since those files are not used anywhere.
Additionally, every additional package raises the security risks since it raises the amount of code
around, also this is nothing python specific.
Since python-3 is totally optional, it should be an option to pull it in, not a forced pull, where
users have to know, that is is optional and could be masked.

-A news item, which is only shown, once python-3* is installed.

It is only shown _after_ installation of python-3.
It just suggests to not set python-3 as main python version and to run the python-updater, but it
does not tell the user, that python-3 is still completly optional and not needed for him.

-Arfrever also said, he would add a seperate news item, when python-3 gets stabilized.

Now the stabilisation bug is open, the first arch is stabilized and i still dont see any news item,
which does prepare the users in advance.


Beside those points, one additional main issue is, that i and others dont seem to be able to have a
discussion with Arfrever about this topics. He says, he has no time for it or says, that he already
had shown arguments, but cannot show any evidence or just stops responding without any note.

Even if all those changes would have good reasons for them, the way, how they are done and
communicated is not very well chosen. And since i dont seem to be able to get any discussion with
Arfrever about those points, i will also CC devrel. For now, to inform them and, also in the hope,
that it is not needed and those issues can be resolved, in preparation for a discussion and decision
on those topics, if needed later one.


So for Arfrever: I also CCed you, so that i can be sure, that you get the mail. Please answer to all
of my above points with arguments. Choose whatever way you prefer for that (public mail, private
mail, public IRC discussion or private message via IRC). If you missed some points or others appear,
i will answer and ask about those.
If you do not answer at all or do not answer with arguments to my satisfaction within 14 days, i
will escalate those issues to devrel.

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Gilles Dartiguelongue]

Le jeudi 27 mai 2010 à 16:33 +0200, Thomas Sachau a écrit :
> Hi together,
> 
> since i am not able to get any real argument or even discussion on IRC nor on this mailing list from
> Arfrever (main person behind those changes), i would like to raise the following points now on this
> mailing list as told on IRC, so he gets the chance to answer those points and to clear the issues:
> 
> -major changes to python eclasses have been done without peer review on this mailing list. This
> includes pulling in python-3* versions, even when they are not required nor used on the user system

python3 being pulled has already been discussed on this mailing list.
Any ebuild that has a >=python-2 dependency will pull python3 (at least
with portage).

-- 
Gilles Dartiguelongue <eva@gentoo.org>
Gentoo

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 2528 bytes --]

2010-05-27 16:33:39 Thomas Sachau napisał(a):
> Hi together,
> 
> since i am not able to get any real argument or even discussion on IRC nor on this mailing list from
> Arfrever (main person behind those changes), i would like to raise the following points now on this
> mailing list as told on IRC, so he gets the chance to answer those points and to clear the issues:
> 
> -major changes to python eclasses have been done without peer review on this mailing list.

There weren't any major changes in python.eclass in last months. In r1.96:r1.100 I committed a set of minor changes.
Some code was moved, which might cause false impression that there were some major changes:
- "MISCELLANEOUS FUNCTIONS" section (with 8 functions) was moved to before "FUNCTIONS FOR PACKAGES SUPPORTING
   INSTALLATION FOR MULTIPLE PYTHON ABIS" section.
- python_pkg_setup() function was moved to "MISCELLANEOUS FUNCTIONS" section.
- A part of python_mod_cleanup() function was split to _python_clean_compiled_modules() function.
  python_mod_cleanup() now calls _python_clean_compiled_modules().
- 2 loops in python_mod_optimize() were divided and some code was reindented.

> This includes pulling in python-3* versions

It's untruth. python.eclass or distutils.eclass don't enforce any particular version of Python.

> -A news item, which is only shown, once python-3* is installed.
> 
> It is only shown _after_ installation of python-3.

You might file an enhancement request for Portage to show news items before installation.

> -Arfrever also said, he would add a seperate news item, when python-3 gets stabilized.

I never said it. The plan was to have 1 news item for all users, who have installed Python 3.1.

> Beside those points, one additional main issue is, that i and others dont seem to be able to have a
> discussion with Arfrever about this topics.

I'm answering to new (unanswered), rational arguments about these topics.

> He says, he has no time for it or says, that he already had shown arguments, but cannot show
> any evidence or just stops responding without any note.

There had been multiple threads about Python 3. Read e-mails written by me or Dirkjan Ochtman in the following threads:
http://archives.gentoo.org/gentoo-dev/msg_115ce6fa1a09de286bf58db12df463c6.xml
http://archives.gentoo.org/gentoo-dev/msg_814e67764c17f88bde94f22e9a392e4f.xml
http://archives.gentoo.org/gentoo-dev/msg_2591c1b9a7e7b72383d3841bc05dc417.xml

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 4377 bytes --]

Am 05.06.2010 16:43, schrieb Arfrever Frehtes Taifersar Arahesis:
> 2010-05-27 16:33:39 Thomas Sachau napisał(a):
>> Hi together,
>>
>> since i am not able to get any real argument or even discussion on IRC nor on this mailing list from
>> Arfrever (main person behind those changes), i would like to raise the following points now on this
>> mailing list as told on IRC, so he gets the chance to answer those points and to clear the issues:
>>
>> -major changes to python eclasses have been done without peer review on this mailing list.
> 
> There weren't any major changes in python.eclass in last months. In r1.96:r1.100 I committed a set of minor changes.
> Some code was moved, which might cause false impression that there were some major changes:
> - "MISCELLANEOUS FUNCTIONS" section (with 8 functions) was moved to before "FUNCTIONS FOR PACKAGES SUPPORTING
>    INSTALLATION FOR MULTIPLE PYTHON ABIS" section.
> - python_pkg_setup() function was moved to "MISCELLANEOUS FUNCTIONS" section.
> - A part of python_mod_cleanup() function was split to _python_clean_compiled_modules() function.
>   python_mod_cleanup() now calls _python_clean_compiled_modules().
> - 2 loops in python_mod_optimize() were divided and some code was reindented.

You dont call the support for multiple python slots a "major change"? I did not see any RFC about
your idea, concept or implementation.

Even when other people, after the implementation was done, tried to talk with you about your way and
pointing out other possible ways, which would be less complex (e.g. the "ruby way" or implementation
on package manager side), you did not listen or refused to accept them because some optional
additional feature might not work with that way.

> 
>> This includes pulling in python-3* versions
> 
> It's untruth. python.eclass or distutils.eclass don't enforce any particular version of Python.

If any package does inherit python or distutils eclass, then those eclasses do pull in
"dev-lang/python", which is unversioned, so it will always pull in the latest version, in this case
python-3*. You could change this, so it allows any major installed slot to satisfy the python
dependency.

Additionally, your implementation pulls in python-3* for packages, which support multiple versions
of python, but would work fine with the already installed python-2* version. This behaviour might be
fine, when one could actually use python-3* as main version, but currently you pull in an unneeded
and unused python slot.

And your current implementation requires the use of another tool, python-updater, to maintain python
related ebuilds, since your current implementation does not allow the package manager to see the
needed/used dependencies and to (re)compile the packages as needed.

> 
>> -A news item, which is only shown, once python-3* is installed.
>>
>> It is only shown _after_ installation of python-3.
> 
> You might file an enhancement request for Portage to show news items before installation.

Huh? Your news item explicitly requires the installation of python-3, before it is shown. This has
nothing to do with the package manager, it is, what you define in your news item.

> 
>> Beside those points, one additional main issue is, that i and others dont seem to be able to have a
>> discussion with Arfrever about this topics.
> 
> I'm answering to new (unanswered), rational arguments about these topics.
> 
>> He says, he has no time for it or says, that he already had shown arguments, but cannot show
>> any evidence or just stops responding without any note.
> 
> There had been multiple threads about Python 3. Read e-mails written by me or Dirkjan Ochtman in the following threads:
> http://archives.gentoo.org/gentoo-dev/msg_115ce6fa1a09de286bf58db12df463c6.xml
> http://archives.gentoo.org/gentoo-dev/msg_814e67764c17f88bde94f22e9a392e4f.xml
> http://archives.gentoo.org/gentoo-dev/msg_2591c1b9a7e7b72383d3841bc05dc417.xml

I did read the threads about python-3, but i still dont see any real argument, why you want everyone
to install python-3*, also it is not required, not needed and not used. Also, you have written, that
the python team does not suggest to mask python-3, but did not give any reason for that. So could
you do that now?


-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Harald van Dijk]

On Sat, Jun 05, 2010 at 05:49:08PM +0200, Thomas Sachau wrote:
> If any package does inherit python or distutils eclass, then those eclasses do pull in
> "dev-lang/python", which is unversioned, so it will always pull in the latest version, in this case
> python-3*. You could change this, so it allows any major installed slot to satisfy the python
> dependency.

A dependency on dev-lang/python *is* satisfied by any slot, any version. You've
been told so already, if I recall correctly.

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 889 bytes --]

Am 05.06.2010 20:31, schrieb Harald van Dijk:
> On Sat, Jun 05, 2010 at 05:49:08PM +0200, Thomas Sachau wrote:
>> If any package does inherit python or distutils eclass, then those eclasses do pull in
>> "dev-lang/python", which is unversioned, so it will always pull in the latest version, in this case
>> python-3*. You could change this, so it allows any major installed slot to satisfy the python
>> dependency.
> 
> A dependency on dev-lang/python *is* satisfied by any slot, any version. You've
> been told so already, if I recall correctly.
> 
> 

Every slot and every version *should* satisfy a "dev-lang/python" dependency, but currently such
unspecified version dependency does automaticly pull in the latest version/slot, which in case of
python does mean python-3*, even when you have e.g. python:2.6 installed.

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Harald van Dijk]

On Sun, Jun 06, 2010 at 01:03:48AM +0200, Thomas Sachau wrote:
> Am 05.06.2010 20:31, schrieb Harald van Dijk:
> > On Sat, Jun 05, 2010 at 05:49:08PM +0200, Thomas Sachau wrote:
> >> If any package does inherit python or distutils eclass, then those eclasses do pull in
> >> "dev-lang/python", which is unversioned, so it will always pull in the latest version, in this case
> >> python-3*. You could change this, so it allows any major installed slot to satisfy the python
> >> dependency.
> > 
> > A dependency on dev-lang/python *is* satisfied by any slot, any version. You've
> > been told so already, if I recall correctly.
> 
> Every slot and every version *should* satisfy a "dev-lang/python" dependency, but currently such
> unspecified version dependency does automaticly pull in the latest version/slot, which in case of
> python does mean python-3*, even when you have e.g. python:2.6 installed.

Fine, I'll be as explicit as possible: not quite. I have a Python 3-free system. I created
a dummy ebuild that does nothing but pull in unversioned python. Let's
see how it behaves.

$ emerge -pv python

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  NS   ] dev-lang/python-3.1.2-r3 [2.6.5-r2] USE="gdbm ipv6 ncurses readline sqlite ssl threads tk (wide-unicode) xml -build -doc -examples -wininst" ELIBC="(-uclibc)" 9,558 kB

$ cat test-2.0.ebuild 
KEYWORDS="~amd64"
SLOT="0"
DEPEND="dev-lang/python"

$ emerge -pv test

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    ] test/test-2.0  0 kB [1]

Total: 1 package (1 new), Size of downloads: 0 kB
Portage tree and overlays:
 [0] /usr/portage
 [1] /etc/portage/overlay

Note how python 3 is *not* pulled in, despite an unversioned dependency on dev-lang/python.
You only get that if you tell portage to try and update dependencies as
well, and yes, if you do that, it's only fair that it attempts to update python.

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 2973 bytes --]

Am 06.06.2010 01:38, schrieb Harald van Dijk:
> On Sun, Jun 06, 2010 at 01:03:48AM +0200, Thomas Sachau wrote:
>> Am 05.06.2010 20:31, schrieb Harald van Dijk:
>>> On Sat, Jun 05, 2010 at 05:49:08PM +0200, Thomas Sachau wrote:
>>>> If any package does inherit python or distutils eclass, then those eclasses do pull in
>>>> "dev-lang/python", which is unversioned, so it will always pull in the latest version, in this case
>>>> python-3*. You could change this, so it allows any major installed slot to satisfy the python
>>>> dependency.
>>>
>>> A dependency on dev-lang/python *is* satisfied by any slot, any version. You've
>>> been told so already, if I recall correctly.
>>
>> Every slot and every version *should* satisfy a "dev-lang/python" dependency, but currently such
>> unspecified version dependency does automaticly pull in the latest version/slot, which in case of
>> python does mean python-3*, even when you have e.g. python:2.6 installed.
> 
> Fine, I'll be as explicit as possible: not quite. I have a Python 3-free system. I created
> a dummy ebuild that does nothing but pull in unversioned python. Let's
> see how it behaves.
> 
> $ emerge -pv python
> 
> These are the packages that would be merged, in order:
> 
> Calculating dependencies... done!
> [ebuild  NS   ] dev-lang/python-3.1.2-r3 [2.6.5-r2] USE="gdbm ipv6 ncurses readline sqlite ssl threads tk (wide-unicode) xml -build -doc -examples -wininst" ELIBC="(-uclibc)" 9,558 kB
> 
> $ cat test-2.0.ebuild 
> KEYWORDS="~amd64"
> SLOT="0"
> DEPEND="dev-lang/python"
> 
> $ emerge -pv test
> 
> These are the packages that would be merged, in order:
> 
> Calculating dependencies... done!
> [ebuild  N    ] test/test-2.0  0 kB [1]
> 
> Total: 1 package (1 new), Size of downloads: 0 kB
> Portage tree and overlays:
>  [0] /usr/portage
>  [1] /etc/portage/overlay
> 
> Note how python 3 is *not* pulled in, despite an unversioned dependency on dev-lang/python.
> You only get that if you tell portage to try and update dependencies as
> well, and yes, if you do that, it's only fair that it attempts to update python.
> 
> 

And you do want to update world with all the dependencies of it, so even if it is not pulled in
during installation, it will be pulled in during world update.

Since python-3* is currently useless and not required for any package, the dependency should by
default only pull in python-2* like this:

=dev-lang/python-2*

With that, the default way would not pull in a package, which is not needed or used. And if there
will be any package, which really requires python-3*, it simply requests it in (R)DEPEND of the
ebuild, which then would overwrite the default value of the eclass and pull in python-3*.

Are there any reasons to pull in a package, which is not requested by the user, not required by any
package and by default not used by any package?



-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Sebastian Pipping]

Thomas,


On 06/06/10 04:01, Thomas Sachau wrote:
> [..] so even if it is not pulled in during installation, it will be pulled in during world update.

sounds right.  Preventing this requires either masking or a
dont-pull-uninstalled-slots switch for portage (which I am not
suggesting), right?

So that means Python-3 is pulled in on world update no matter what
version the eclass makes packages depend on, right?


> Since python-3* is currently useless and not required for any package, the dependency should by
> default only pull in python-2* like this:
> 
> =dev-lang/python-2*
> 
> With that, the default way would not pull in a package, which is not needed or used. And if there
> will be any package, which really requires python-3*, it simply requests it in (R)DEPEND of the
> ebuild, which then would overwrite the default value of the eclass and pull in python-3*.

That's an interesting idea.

Leaving potential non-portage Gentoo setups aside for the moment is
there a case of a Gentoo with Python 3.x only where it would be forced
to install 2.x then without reason, too?

I still wonder how it would prevent the update-world-pulls-py3k case.
Would it?  If not, would it still be of help?


> Are there any reasons to pull in a package, which is not requested by the user, not required by any
> package and by default not used by any package?

That a question I haven't seen answered before, either.  Arfrever?



Sebastian

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1711 bytes --]

On Sun, 06 Jun 2010 04:19:28 +0200
Sebastian Pipping <sping@gentoo.org> wrote:

> Thomas,
> 
> 
> On 06/06/10 04:01, Thomas Sachau wrote:
> > [..] so even if it is not pulled in during installation, it will be
> > pulled in during world update.
> 
> sounds right.  Preventing this requires either masking or a
> dont-pull-uninstalled-slots switch for portage (which I am not
> suggesting), right?

In fact, these two seem to be the most reasonable solutions
for the problem. While this switch idea is more universal (and I guess
-- not that hard to implement), masking should be simpler.

> > Since python-3* is currently useless and not required for any
> > package, the dependency should by default only pull in python-2*
> > like this:
> > 
> > =dev-lang/python-2*
> > 
> > With that, the default way would not pull in a package, which is
> > not needed or used. And if there will be any package, which really
> > requires python-3*, it simply requests it in (R)DEPEND of the
> > ebuild, which then would overwrite the default value of the eclass
> > and pull in python-3*.
> 
> That's an interesting idea.

It sounds quite pointless to me. Forcing the packages to assume they
don't support the newer version just because nothing requires it yet?

> > Are there any reasons to pull in a package, which is not requested
> > by the user, not required by any package and by default not used by
> > any package?
> 
> That a question I haven't seen answered before, either.  Arfrever?

It _is_ requested by user. User requested upgrade of all dependant
packages, and here it goes.

-- 
Best regards,
Michał Górny

<http://mgorny.alt.pl>
<xmpp:mgorny@jabber.ru>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Dale]

Michał Górny wrote:
> On Sun, 06 Jun 2010 04:19:28 +0200
> Sebastian Pipping<sping@gentoo.org>  wrote:
>
>    
>> Thomas,
>>
>>
>> On 06/06/10 04:01, Thomas Sachau wrote:
>>      
>>>   not required by any package and by default not used by
>>> any package?
>>>        
>> That a question I haven't seen answered before, either.  Arfrever?
>>      
> It _is_ requested by user. User requested upgrade of all dependant
> packages, and here it goes.
>
>    

So answer the last two parts of that question.  I removed the rest to 
sort of highlight the part you missed.

Dale

:-)  :-)

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 2387 bytes --]

Am 06.06.2010 09:37, schrieb Michał Górny:
> On Sun, 06 Jun 2010 04:19:28 +0200
> Sebastian Pipping <sping@gentoo.org> wrote:
> 
>> Thomas,
>>
>>
>> On 06/06/10 04:01, Thomas Sachau wrote:
>>> Since python-3* is currently useless and not required for any
>>> package, the dependency should by default only pull in python-2*
>>> like this:
>>>
>>> =dev-lang/python-2*
>>>
>>> With that, the default way would not pull in a package, which is
>>> not needed or used. And if there will be any package, which really
>>> requires python-3*, it simply requests it in (R)DEPEND of the
>>> ebuild, which then would overwrite the default value of the eclass
>>> and pull in python-3*.
>>
>> That's an interesting idea.
> 
> It sounds quite pointless to me. Forcing the packages to assume they
> don't support the newer version just because nothing requires it yet?

This is not about forcing a python-2* dependency, it is just about setting a sane default. We still
have many python related packages, which dont work with python-3, but i dont know of packages, which
dont work with python-2. So a sane default would be to require python-2, when nothing else is set in
the ebuild instead of assuming, that it works for every version including python-3.

You can always overwrite this dependency in the ebuild, so you dont force anything.

> 
>>> Are there any reasons to pull in a package, which is not requested
>>> by the user, not required by any package and by default not used by
>>> any package?
>>
>> That a question I haven't seen answered before, either.  Arfrever?
> 
> It _is_ requested by user. User requested upgrade of all dependant
> packages, and here it goes.

Before python-3 got introduced, packages, which only support python-2, did just inherit python or
distutils eclass and did not depend on python, because this dependency was in the eclass. Now with
the introduction of python-3, this dependency string will introduce python-3, also those packages
where not tested with python-3 and probably wont work with it.

As a user, i expect a world update to update/install all needed and required packages. python-3 is
neither required, nor needed or used, it is a complete optional dependency and should be handled
like that, see my other mail with a possible way to handle it.

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Graham Murray]

Thomas Sachau <tommy@gentoo.org> writes:

> Since python-3* is currently useless and not required for any package, the dependency should by
> default only pull in python-2* like this:
>
> =dev-lang/python-2*
>
> With that, the default way would not pull in a package, which is not needed or used. And if there
> will be any package, which really requires python-3*, it simply requests it in (R)DEPEND of the
> ebuild, which then would overwrite the default value of the eclass and pull in python-3*.

That would not work. For example if package 'bar' supports both python-2
and python-3, your mechanism will only build in python-2 support. That
is fine until package 'foo' comes along which uses 'bar' but also
requires python-3 - thus not only must python-3 be pulled in as a result
of foo's (R)DEPEND but also 'bar' must be rebuilt with python-3 support.

This could be done by adding python2 and python3 USE flags to packages
which support both and only have python2 enabled by default. Then 'foo'
could have a conditional (R)DEPEMD on 'bar[python3]', but that would
require the user to change the USE flags whereas the current system
handles everything automatically.

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 3828 bytes --]

Am 06.06.2010 08:36, schrieb Graham Murray:
> Thomas Sachau <tommy@gentoo.org> writes:
> 
>> Since python-3* is currently useless and not required for any package, the dependency should by
>> default only pull in python-2* like this:
>>
>> =dev-lang/python-2*
>>
>> With that, the default way would not pull in a package, which is not needed or used. And if there
>> will be any package, which really requires python-3*, it simply requests it in (R)DEPEND of the
>> ebuild, which then would overwrite the default value of the eclass and pull in python-3*.
> 
> That would not work. For example if package 'bar' supports both python-2
> and python-3, your mechanism will only build in python-2 support. That
> is fine until package 'foo' comes along which uses 'bar' but also
> requires python-3 - thus not only must python-3 be pulled in as a result
> of foo's (R)DEPEND but also 'bar' must be rebuilt with python-3 support.

And that is, how it should be done. Think for example of a package foo, which has a "png" USE flag,
which will pull in libpng. By default disabled, it results in disabled png support and libpng not
pulled in. Now, when some other package requires foo with png USE flag enabled, you will have to
recompile foo with png USE flag and you will have to install libpng. We have the usedeps of EAPI-2
for it and the package manager does handle it, depending on the user requests.

But for optional python-3 support, you cannot control it easily with your package manager because of
the current way, how it is implemented.

> 
> This could be done by adding python2 and python3 USE flags to packages
> which support both and only have python2 enabled by default. Then 'foo'
> could have a conditional (R)DEPEMD on 'bar[python3]', but that would
> require the user to change the USE flags whereas the current system
> handles everything automatically.

And automagic behind the scene is exactly, what should not happen. The user should be able to
control optional dependencies, it should be the same for python like for any other package.

My base proposal for this is something like this:

Every package defines the language(s), where it could be installed for multiple slots, e.g.:

MULTI_SLOT="python" or
MULTI_SLOT="python ruby"

Additionally, it should define the supported slots, something like this:

SUPPORTED_RUBY_SLOTS="1.8 1.9" or
SUPPORTED_PYTHON_SLOTS="2.5 2.6 3.0 3.1"

Now the package manager should take those vars and convert them to some expanded USE vars like:

RUBY_SLOTS="1.8 1.9" or
PYTHON_SLOTS="2.5 2.6 3.0 3.1"

By default, the current active version should be enabled, the others disabled. In addition, every
dependency, which requires ruby/python should get internal usedeps, so that they require the same
slots as this package. Every enabled slot should of course pull in that slot of the language.

With this way, every user can select, which slots he wants to use and which slots should be
installed. And if any package requires an installed version for a specific slot, it can be set in
(R)DEPEND (e.g. like DEPEND=" python? ( dev-db/foo[pyhon_slot_2.6] )")

It would also reduce the amount of code, since we would not have to implement multi-slot support in
many different eclasses and it would additionally move the dependency control back to the package
manager unlike the current python implementation.

I currently have a branch of portage ("multilib-portage"), which can install a package for different
platforms, it could be extended to implement the above idea and i plan to do that. Since i am the
only person working on it and me only having limited time and knowledge, it could still take some
time, if noone else wants to step in and help with it.

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Matti Bickel]

[-- Attachment #1: Type: text/plain, Size: 819 bytes --]

On 06/06/2010 12:40 PM, Thomas Sachau wrote:
> My base proposal for this is something like this:
> 
> Every package defines the language(s), where it could be installed for multiple slots, e.g.:
> 
> MULTI_SLOT="python" or
> MULTI_SLOT="python ruby"
> 
> Additionally, it should define the supported slots, something like this:
> 
> SUPPORTED_RUBY_SLOTS="1.8 1.9" or
> SUPPORTED_PYTHON_SLOTS="2.5 2.6 3.0 3.1"

Don't get me wrong, but isn't that what the python developers guide[1]
says? ("python.eclass supports PYTHON_DEPEND  helper variable, which
allows to specify minimal and maximal version of Python.")

I thought the whole point of this debate was that nobody cared enough to
convert all those ebuilds to use PYTHON_DEPEND properly. The proper fix
is to convert ebuilds to the new syntax.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 2167 bytes --]

Am 06.06.2010 13:09, schrieb Matti Bickel:
> On 06/06/2010 12:40 PM, Thomas Sachau wrote:
>> My base proposal for this is something like this:
>>
>> Every package defines the language(s), where it could be installed for multiple slots, e.g.:
>>
>> MULTI_SLOT="python" or
>> MULTI_SLOT="python ruby"
>>
>> Additionally, it should define the supported slots, something like this:
>>
>> SUPPORTED_RUBY_SLOTS="1.8 1.9" or
>> SUPPORTED_PYTHON_SLOTS="2.5 2.6 3.0 3.1"
> 
> Don't get me wrong, but isn't that what the python developers guide[1]
> says? ("python.eclass supports PYTHON_DEPEND  helper variable, which
> allows to specify minimal and maximal version of Python.")
> 
> I thought the whole point of this debate was that nobody cared enough to
> convert all those ebuilds to use PYTHON_DEPEND properly. The proper fix
> is to convert ebuilds to the new syntax.
> 

The current python eclass also uses some vars to specify the supported slots, yes, but it is more
complex and harder to maintain in addition to the fact, that the dependency part is hidden from the
package manager.

I dont think, that you can tell portage with the current implementation, that it should only install
python modules for python-2.6 by default and additionally python modules for python-3.1 for selected
packages. Portage will also install newer slots of python, even when the user does not request them
and no package requires them, which will result in unneeded and unused versions on disk.

And if you add a python slot or remove one, portage currently is not able to see that and to
reinstall packages, which had modules installed for that slot. You need another tool
(python-updater) to check that and to call the needed reinstalls.

With my solution, there are only modules installed for selected slots. And if you have selected a
slot, the related python version is pulled in by portage. If you disable that slot, you can
reinstall those packages with --newuse option and then can remove that python slot with --depclean.
No need for another tool, simple handling by the package manager

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Domen Kožar]

[-- Attachment #1: Type: text/plain, Size: 1728 bytes --]

> The current python eclass also uses some vars to specify the supported slots, yes, but it is more
> complex and harder to maintain in addition to the fact, that the dependency part is hidden from the
> package manager.
> 
> I dont think, that you can tell portage with the current implementation, that it should only install
> python modules for python-2.6 by default and additionally python modules for python-3.1 for selected
> packages. Portage will also install newer slots of python, even when the user does not request them
> and no package requires them, which will result in unneeded and unused versions on disk.

Beg my pardon, but that is untrue AFAIK.

Portage will install packages only for active python version, unless
USE_PYTHON is set.

> And if you add a python slot or remove one, portage currently is not able to see that and to
> reinstall packages, which had modules installed for that slot. You need another tool
> (python-updater) to check that and to call the needed reinstalls.

I agree with this fact, user should not be required to read additional
documenation for portage to function as wanted. 

I'm very unfamiliar with inner workings of portage, but using
python-updater implementation, USE_PYTHON behaviour shouldn't be that
hard to implement?

> 
> With my solution, there are only modules installed for selected slots. And if you have selected a
> slot, the related python version is pulled in by portage. If you disable that slot, you can
> reinstall those packages with --newuse option and then can remove that python slot with --depclean.
> No need for another tool, simple handling by the package manager
Explicit is **** than implicit:)


Cheers, Domen

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 1043 bytes --]

Am 06.06.2010 13:50, schrieb Domen Kožar:
>> The current python eclass also uses some vars to specify the supported slots, yes, but it is more
>> complex and harder to maintain in addition to the fact, that the dependency part is hidden from the
>> package manager.
>>
>> I dont think, that you can tell portage with the current implementation, that it should only install
>> python modules for python-2.6 by default and additionally python modules for python-3.1 for selected
>> packages. Portage will also install newer slots of python, even when the user does not request them
>> and no package requires them, which will result in unneeded and unused versions on disk.
> 
> Beg my pardon, but that is untrue AFAIK.
> 
> Portage will install packages only for active python version, unless
> USE_PYTHON is set.

And by default, there is an active version for python-2 and an active version for python-3, which
means currently an install for e.g. python:2.6 and python:3.1.

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 956 bytes --]

Am 06.06.2010 13:50, schrieb Domen Kožar:
>> And if you add a python slot or remove one, portage currently is not able to see that and to
>> reinstall packages, which had modules installed for that slot. You need another tool
>> (python-updater) to check that and to call the needed reinstalls.
> 
> I agree with this fact, user should not be required to read additional
> documenation for portage to function as wanted. 
> 
> I'm very unfamiliar with inner workings of portage, but using
> python-updater implementation, USE_PYTHON behaviour shouldn't be that
> hard to implement?

You want some additional switch to portage, which does the work of python-updater? That would just
move the code, but would still have the same limitations. What does speak against explicit user
control for optional features/slots, including dependency handling by the package manager like in my
proposal?

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Domen Kožar]

[-- Attachment #1: Type: text/plain, Size: 1259 bytes --]

On Sun, 2010-06-06 at 14:41 +0200, Thomas Sachau wrote:
> Am 06.06.2010 13:50, schrieb Domen Kožar:
> >> And if you add a python slot or remove one, portage currently is not able to see that and to
> >> reinstall packages, which had modules installed for that slot. You need another tool
> >> (python-updater) to check that and to call the needed reinstalls.
> > 
> > I agree with this fact, user should not be required to read additional
> > documenation for portage to function as wanted. 
> > 
> > I'm very unfamiliar with inner workings of portage, but using
> > python-updater implementation, USE_PYTHON behaviour shouldn't be that
> > hard to implement?
> 
> You want some additional switch to portage, which does the work of python-updater? That would just
> move the code, but would still have the same limitations. What does speak against explicit user
> control for optional features/slots, including dependency handling by the package manager like in my
> proposal?
> 

Maybe I expressed myself wrong. Portage would only reuse python-updater
to detect and repair changes with python installation.

If I understand correctly, one solution would be to pull stable 2.x, and
only install other slots according to USE_PYTHON.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 1860 bytes --]

Am 06.06.2010 15:35, schrieb Domen Kožar:
> On Sun, 2010-06-06 at 14:41 +0200, Thomas Sachau wrote:
>> Am 06.06.2010 13:50, schrieb Domen Kožar:
>>>> And if you add a python slot or remove one, portage currently is not able to see that and to
>>>> reinstall packages, which had modules installed for that slot. You need another tool
>>>> (python-updater) to check that and to call the needed reinstalls.
>>>
>>> I agree with this fact, user should not be required to read additional
>>> documenation for portage to function as wanted. 
>>>
>>> I'm very unfamiliar with inner workings of portage, but using
>>> python-updater implementation, USE_PYTHON behaviour shouldn't be that
>>> hard to implement?
>>
>> You want some additional switch to portage, which does the work of python-updater? That would just
>> move the code, but would still have the same limitations. What does speak against explicit user
>> control for optional features/slots, including dependency handling by the package manager like in my
>> proposal?
>>
> 
> Maybe I expressed myself wrong. Portage would only reuse python-updater
> to detect and repair changes with python installation.
> 
> If I understand correctly, one solution would be to pull stable 2.x, and
> only install other slots according to USE_PYTHON.
> 

And how would that improve the current implementation?

If you only reuse python-updater code inside portage, the issues of the current implementation still
remain. And you dont seem to answer my question.

Why dont you want to allow the user to control *per package*, for which slots it should be installed?
And why dont you want to allow the package manager to mangle the dependency part with already
existing code ( USE flags and --newuse, dependency tree and --depclean)?


-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1784 bytes --]

On Sun, Jun 06, 2010 at 01:35:55PM +0000, Domen Koooar wrote:
> On Sun, 2010-06-06 at 14:41 +0200, Thomas Sachau wrote:
> > Am 06.06.2010 13:50, schrieb Domen Kožar:
> > >> And if you add a python slot or remove one, portage currently is not able to see that and to
> > >> reinstall packages, which had modules installed for that slot. You need another tool
> > >> (python-updater) to check that and to call the needed reinstalls.
> > > 
> > > I agree with this fact, user should not be required to read additional
> > > documenation for portage to function as wanted. 
> > > 
> > > I'm very unfamiliar with inner workings of portage, but using
> > > python-updater implementation, USE_PYTHON behaviour shouldn't be that
> > > hard to implement?
> > 
> > You want some additional switch to portage, which does the work of python-updater? That would just
> > move the code, but would still have the same limitations. What does speak against explicit user
> > control for optional features/slots, including dependency handling by the package manager like in my
> > proposal?
> > 
> 
> Maybe I expressed myself wrong. Portage would only reuse python-updater
> to detect and repair changes with python installation.
> 
> If I understand correctly, one solution would be to pull stable 2.x, and
> only install other slots according to USE_PYTHON.

$PACKAGE_MANAGER should not have to use python-updater *period*.  If 
the USE_EXPAND route was taken for desired python versions, mapped 
down to virtual/python:$SLOT, the manager would know automatically of 
the needed python subgraph dependency wise.

Really wish people would stop pointing at python-updater; it's a flat 
out hack that exists only due to info not being exported to the PM.  

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 519 bytes --]

2010-06-06 12:40:28 Thomas Sachau napisał(a):
> Additionally, it should define the supported slots, something like this:
> 
> SUPPORTED_RUBY_SLOTS="1.8 1.9" or
> SUPPORTED_PYTHON_SLOTS="2.5 2.6 3.0 3.1"
> 
> Now the package manager should take those vars and convert them to some expanded USE vars like:
> 
> RUBY_SLOTS="1.8 1.9" or
> PYTHON_SLOTS="2.5 2.6 3.0 3.1"

We are already working on automatic generation of USE flags in python.eclass (in newer EAPIs).

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

Am 06.06.2010 15:44, schrieb Arfrever Frehtes Taifersar Arahesis:
> 2010-06-06 12:40:28 Thomas Sachau napisał(a):
>> Additionally, it should define the supported slots, something like this:
>>
>> SUPPORTED_RUBY_SLOTS="1.8 1.9" or
>> SUPPORTED_PYTHON_SLOTS="2.5 2.6 3.0 3.1"
>>
>> Now the package manager should take those vars and convert them to some expanded USE vars like:
>>
>> RUBY_SLOTS="1.8 1.9" or
>> PYTHON_SLOTS="2.5 2.6 3.0 3.1"
> 
> We are already working on automatic generation of USE flags in python.eclass (in newer EAPIs).
> 

And why do you want to implement such code in every eclass?

Whats wrong with implementing it on the package manager side once and reusing it for every
eclass/ebuild, which needs such code?

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 1336 bytes --]

2010-06-06 15:54:23 Thomas Sachau napisał(a):
> Am 06.06.2010 15:44, schrieb Arfrever Frehtes Taifersar Arahesis:
> > 2010-06-06 12:40:28 Thomas Sachau napisał(a):
> >> Additionally, it should define the supported slots, something like this:
> >>
> >> SUPPORTED_RUBY_SLOTS="1.8 1.9" or
> >> SUPPORTED_PYTHON_SLOTS="2.5 2.6 3.0 3.1"
> >>
> >> Now the package manager should take those vars and convert them to some expanded USE vars like:
> >>
> >> RUBY_SLOTS="1.8 1.9" or
> >> PYTHON_SLOTS="2.5 2.6 3.0 3.1"
> > 
> > We are already working on automatic generation of USE flags in python.eclass (in newer EAPIs).
> > 
> 
> And why do you want to implement such code in every eclass?
> 
> Whats wrong with implementing it on the package manager side once and reusing it for every
> eclass/ebuild, which needs such code?

We can accept such an implementation if it meets all our requirements.
E.g. Python ebuilds must specify list of NOT supported Python ABIs (e.g. using whitespace-separated
list of fnmatch patterns). When a new slot of Python is available, then updating one place
(an eclass or a file in profiles) must be sufficient to add a USE flag to all ebuilds (which
don't specify a range of not supported Python ABIs which would match the new Python ABI).

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Alec Warner]

On Sun, Jun 6, 2010 at 6:54 AM, Thomas Sachau <tommy@gentoo.org> wrote:
> Am 06.06.2010 15:44, schrieb Arfrever Frehtes Taifersar Arahesis:
>> 2010-06-06 12:40:28 Thomas Sachau napisał(a):
>>> Additionally, it should define the supported slots, something like this:
>>>
>>> SUPPORTED_RUBY_SLOTS="1.8 1.9" or
>>> SUPPORTED_PYTHON_SLOTS="2.5 2.6 3.0 3.1"
>>>
>>> Now the package manager should take those vars and convert them to some expanded USE vars like:
>>>
>>> RUBY_SLOTS="1.8 1.9" or
>>> PYTHON_SLOTS="2.5 2.6 3.0 3.1"
>>
>> We are already working on automatic generation of USE flags in python.eclass (in newer EAPIs).
>>
>
> And why do you want to implement such code in every eclass?
>
> Whats wrong with implementing it on the package manager side once and reusing it for every
> eclass/ebuild, which needs such code?

I don't think arfrever thinks there is anything wrong.  The main
problem with implementing things in a package manager is time.  Why
have a big long discussion about something that takes years to agree
on, implement, and then get into an approved stable EAPI when you can
just stick things in your eclass and use them in a few weeks / months
(this can be read as a mockery of what was done; I'm not mocking.
Moving quickly is important in many cases and iteration of ideas and
schemes are good.)

I'm all for generalizing the current implementation where it makes
sense; but I'm kinda tired of people bashing it because its not
perfect; I don't think we can necessarily wait for 'perfectly
designed' things every time (no matter what many implementors think.)

-A

>
> --
> Thomas Sachau
>
> Gentoo Linux Developer
>
>

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Hans de Graaff]

[-- Attachment #1: Type: text/plain, Size: 1808 bytes --]

On Sun, 2010-06-06 at 12:40 +0200, Thomas Sachau wrote:

> Every package defines the language(s), where it could be installed for multiple slots, e.g.:
> 
> MULTI_SLOT="python" or
> MULTI_SLOT="python ruby"
> 
> Additionally, it should define the supported slots, something like this:
> 
> SUPPORTED_RUBY_SLOTS="1.8 1.9" or
> SUPPORTED_PYTHON_SLOTS="2.5 2.6 3.0 3.1"
> 
> Now the package manager should take those vars and convert them to some expanded USE vars like:
> 
> RUBY_SLOTS="1.8 1.9" or
> PYTHON_SLOTS="2.5 2.6 3.0 3.1"
> 
> By default, the current active version should be enabled, the others disabled. In addition, every
> dependency, which requires ruby/python should get internal usedeps, so that they require the same
> slots as this package. Every enabled slot should of course pull in that slot of the language.

Something like this is already implemented for ruby. See our
ruby-ng.eclass for details. Note that thinking in terms of slots here is
too limiting. For ruby we currently have four targets, two of which are
slotted versions of the same implementation, and two of which are
independent implementations.

> I currently have a branch of portage ("multilib-portage"), which can install a package for different
> platforms, it could be extended to implement the above idea and i plan to do that. Since i am the
> only person working on it and me only having limited time and knowledge, it could still take some
> time, if noone else wants to step in and help with it.

It might be nice to have generic support for this, especially in light
of packages that have bindings in multiple languages. Please have a look
at ruby-ng.eclass to see our current requirements and feel free to drop
by in #gentoo-ruby for further discussion if needed.

Hans


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Sebastian Pipping]

Thomas,


On 06/06/10 01:04, Thomas Sachau wrote:
> Every slot and every version *should* satisfy a "dev-lang/python" dependency, but currently such
> unspecified version dependency does automaticly pull in the latest version/slot, which in case of
> python does mean python-3*, even when you have e.g. python:2.6 installed.

can you explain how that happens?



Sebastian

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Rémi Cardona]

Le 06/06/2010 02:08, Sebastian Pipping a écrit :
> can you explain how that happens?

Standard dependency resolution for packages with slots, there's nothing
specific about python.

Cheers,

Rémi

Re: [gentoo-dev] Actions of python team, especially Arfrever wrt python eclass and python-3*

[Sebastian Pipping]

Thomas, Arfrever,


the way this discussion currently works does not look fruitful to me.
Questions I would like to raise:

 - How come you don't agree on facts?

 - Is this about processes or results?

 - Have you tried to understand the other side's problem?

 - Could you try a question-explanation of conversation style, instead?

Best,



Sebastian