[gentoo-dev] Python-3.2-related changes

[gentoo-dev] Python-3.2-related changes

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 2153 bytes --]

It is planned that ebuilds of Python 3.2 will automatically set Python 3.2 as main active
version of Python. (Currently ebuilds of Python 3.1 do not automatically set Python 3.1
as main active version of Python.) Final release of Python 3.2 is currently scheduled on
2010-12-11, so you have over 10 months for adjusting of your ebuilds.

The following changes should be made in ebuilds of packages supporting only Python 2:
- Dependency on Python 2 should be set correctly. You can specify it directly in
  {,R}DEPEND or use PYTHON_DEPEND.

    Example:
      PYTHON_DEPEND="2"

      inherit python

- Miscellaneous actions should be executed with correct version of Python.

  - In ebuilds of packages supporting installation for multiple versions of Python,
    please remember to disable installation for all versions of Python 3.
    (Please do not use python_set_active_version() in ebuilds of these packages.)

    Example:
      RESTRICT_PYTHON_ABIS="3.*"

  - In ebuilds of packages not supporting installation for multiple versions of Python,
    set active version of Python using python_set_active_version().
    (Please avoid passing precisely specified Python ABI to python_set_active_version().)

    Example:
      pkg_setup() {
          python_set_active_version 2
      }

    (If given package currently cannot be installed for multiple versions of Python,
    but it is theoretically possible (e.g. this package installs .py files into
    /usr/lib/pythonX.Y/site-packages and does not install any non-versioned libraries
    into /usr/lib), then please try to adjust ebuilds of this package to allow
    installation for multiple versions of Python.)

It is recommended to have set a version of Python 3 as main active version of Python
during testing of these changes to ensure that no required change has been missed.

I consider filing bugs for not adjusted packages after some months (e.g in summer).
(Please note that these changes are not required before stabilization of Python 3.1,
so stabilization of Python 3.1 will not be delayed.)

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 504 bytes --]

2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
> I consider filing bugs for not adjusted packages after some months (e.g in summer).

1123 packages (440 in dev-python category) from 108 categories specify dev-lang/python
or virtual/python in DEPEND or RDEPEND, so actually it might be better to start filing
bugs in this month. If there are no objections, then I would like to file 1 bug per
category (except dev-python category).

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Mark Loeser]

Arfrever Frehtes Taifersar Arahesis <Arfrever@gentoo.org> said:
> 2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
> > I consider filing bugs for not adjusted packages after some months (e.g in summer).
> 
> 1123 packages (440 in dev-python category) from 108 categories specify dev-lang/python
> or virtual/python in DEPEND or RDEPEND, so actually it might be better to start filing
> bugs in this month. If there are no objections, then I would like to file 1 bug per
> category (except dev-python category).

Make trackers and make one bug per package.  Its way too hard to follow
a huge metabug with a bunch of packages listed in it.  Also, I think the
concerns and suggestions that were brought up about the syntax of this
new variable should be addressed first and not ignored.

Thanks,

-- 
Mark Loeser
email         -   halcy0n AT gentoo DOT org
email         -   mark AT halcy0n DOT com
web           -   http://www.halcy0n.com

Re: [gentoo-dev] Python-3.2-related changes

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 1197 bytes --]

2010-02-06 17:54:10 Mark Loeser napisał(a):
> Arfrever Frehtes Taifersar Arahesis <Arfrever@gentoo.org> said:
> > 2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
> > > I consider filing bugs for not adjusted packages after some months (e.g in summer).
> > 
> > 1123 packages (440 in dev-python category) from 108 categories specify dev-lang/python
> > or virtual/python in DEPEND or RDEPEND, so actually it might be better to start filing
> > bugs in this month. If there are no objections, then I would like to file 1 bug per
> > category (except dev-python category).
> 
> Make trackers and make one bug per package.  Its way too hard to follow
> a huge metabug with a bunch of packages listed in it.

Average number of non-dev-python packages handled in 1 bug would be only 6.4, but I can
create create 1 bug per package, if you still want it.

> Also, I think the concerns and suggestions that were brought up about the syntax
> of this new variable should be addressed first and not ignored.

I will include examples in documentation of PYTHON_DEPEND to avoid misunderstanding of
syntax of this variable.

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Thilo Bangert]

[-- Attachment #1: Type: Text/Plain, Size: 1238 bytes --]

Arfrever Frehtes Taifersar Arahesis <Arfrever@gentoo.org> said:
> 2010-02-06 17:54:10 Mark Loeser napisał(a):
> > Arfrever Frehtes Taifersar Arahesis <Arfrever@gentoo.org> said:
> > > 2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
> > > > I consider filing bugs for not adjusted packages after some
> > > > months (e.g in summer).
> > >
> > > 1123 packages (440 in dev-python category) from 108 categories
> > > specify dev-lang/python or virtual/python in DEPEND or RDEPEND, so
> > > actually it might be better to start filing bugs in this month. If
> > > there are no objections, then I would like to file 1 bug per
> > > category (except dev-python category).
> >
> > Make trackers and make one bug per package.  Its way too hard to
> > follow a huge metabug with a bunch of packages listed in it.
> 
> Average number of non-dev-python packages handled in 1 bug would be
>  only 6.4, but I can create create 1 bug per package, if you still want
>  it.

having done mass-filings with one bug per package in the past i know how 
tedious these are.  nevertheless, 1 package per bug it must be - it makes 
all kinds of stuff way easier (think of retirements).

good luck and thanks.
Thilo

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: Python-3.2-related changes

[Duncan]

Thilo Bangert posted on Tue, 09 Feb 2010 09:10:26 +0100 as excerpted:

> having done mass-filings with one bug per package in the past i know how
> tedious these are.  nevertheless, 1 package per bug it must be - it
> makes all kinds of stuff way easier (think of retirements).

What about borrowing the boilerplate scripts flameeyes uses for his 
apparently almost totally automated tinderbox bugs?  Change the various 
strings as appropriate, plugin a list of packages with the problem, and 
away you go! =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Python-3.2-related changes

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 479 bytes --]

2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
> - Dependency on Python 2 should be set correctly. You can specify it directly in
>   {,R}DEPEND or use PYTHON_DEPEND.
> 
>     Example:
>       PYTHON_DEPEND="2"
> 
>       inherit python

It might be useful to provide another example:
If given package requires e.g. Python >=2.6, but does not support Python 3, then you can
set PYTHON_DEPEND="2:2.6".

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

On Sat, Feb 06, 2010 at 12:03:11PM +0100, Arfrever Frehtes Taifersar Arahesis wrote:
> 2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
> > - Dependency on Python 2 should be set correctly. You can specify it directly in
> >   {,R}DEPEND or use PYTHON_DEPEND.
> > 
> >     Example:
> >       PYTHON_DEPEND="2"
> > 
> >       inherit python
> 
> It might be useful to provide another example:
> If given package requires e.g. Python >=2.6, but does not support Python 3, then you can
> set PYTHON_DEPEND="2:2.6".

We really need better syntax than that.  Particular reason you're 
jamming py2k and py3k all into one?  Breaking it down into 
PYTHON2K_DEPEND and PYTHON3K_DEPEND would involve lest magic in 
parsing and would be clear in the reading I'd expect.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 1024 bytes --]

2010-02-06 13:14:41 Brian Harring napisał(a):
> On Sat, Feb 06, 2010 at 12:03:11PM +0100, Arfrever Frehtes Taifersar Arahesis wrote:
> > 2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
> > > - Dependency on Python 2 should be set correctly. You can specify it directly in
> > >   {,R}DEPEND or use PYTHON_DEPEND.
> > > 
> > >     Example:
> > >       PYTHON_DEPEND="2"
> > > 
> > >       inherit python
> > 
> > It might be useful to provide another example:
> > If given package requires e.g. Python >=2.6, but does not support Python 3, then you can
> > set PYTHON_DEPEND="2:2.6".
> 
> We really need better syntax than that.  Particular reason you're 
> jamming py2k and py3k all into one?  Breaking it down into 
> PYTHON2K_DEPEND and PYTHON3K_DEPEND

There's no need for 2 variables when 1 variable is sufficient.

> would involve lest magic in parsing

At least I was able to exercise my bash skills and find a bug in bash :) .

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Markos Chandras]

[-- Attachment #1: Type: Text/Plain, Size: 914 bytes --]

On Saturday 06 February 2010 13:03:11 Arfrever Frehtes Taifersar Arahesis 
wrote:
> 2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
> > - Dependency on Python 2 should be set correctly. You can specify it
> > directly in {,R}DEPEND or use PYTHON_DEPEND.
> >
> >     Example:
> >       PYTHON_DEPEND="2"
> >
> >       inherit python
> 
> It might be useful to provide another example:
> If given package requires e.g. Python >=2.6, but does not support Python 3,
>  then you can set PYTHON_DEPEND="2:2.6".
> 
Could you please update the python documentation [1] or any other online 
documentation you may find useful cause I am pretty sure your recommendations 
in this e-mail will be lost over time.

Many thanks

[1]: http://www.gentoo.org/proj/en/Python/developersguide.xml
-- 
Markos Chandras (hwoarang)
Gentoo Linux Developer
Web: http://hwoarang.silverarrow.org

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 877 bytes --]

2010-02-07 19:43:24 Markos Chandras napisał(a):
> On Saturday 06 February 2010 13:03:11 Arfrever Frehtes Taifersar Arahesis 
> wrote:
> > 2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
> > > - Dependency on Python 2 should be set correctly. You can specify it
> > > directly in {,R}DEPEND or use PYTHON_DEPEND.
> > >
> > >     Example:
> > >       PYTHON_DEPEND="2"
> > >
> > >       inherit python
> > 
> > It might be useful to provide another example:
> > If given package requires e.g. Python >=2.6, but does not support Python 3,
> >  then you can set PYTHON_DEPEND="2:2.6".
> > 
> Could you please update the python documentation [1] or any other online 
> documentation you may find useful cause I am pretty sure your recommendations 
> in this e-mail will be lost over time.

Yes.

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Zac Medico]

On 02/06/2010 03:03 AM, Arfrever Frehtes Taifersar Arahesis wrote:
> 2010-02-05 17:40:00 Arfrever Frehtes Taifersar Arahesis napisał(a):
>> - Dependency on Python 2 should be set correctly. You can specify it directly in
>>   {,R}DEPEND or use PYTHON_DEPEND.
>>
>>     Example:
>>       PYTHON_DEPEND="2"
>>
>>       inherit python
> 
> It might be useful to provide another example:
> If given package requires e.g. Python >=2.6, but does not support Python 3, then you can
> set PYTHON_DEPEND="2:2.6".

I noticed that this generates a depedency like "|| (
=dev-lang/python-2.7* =dev-lang/python-2.6* )" which is very similar
to the way that QT3VERSIONS works in qt3.eclass. One thing that is
sub-optimal about these types of dependencies is that you end up
with lots of installed packages that have out-dated dependencies
when the next minor version of python is released (python-2.8 in
this case). In the case of the python dependencies, it might be more
optimal to use a version range like ">=dev-lang/python-2.6
<dev-lang/python-3".

-- 
Thanks,
Zac

Re: [gentoo-dev] Python-3.2-related changes

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1749 bytes --]

On Sun, Feb 07, 2010 at 12:17:17PM -0800, Zac Medico wrote:
> I noticed that this generates a depedency like "|| (
> =dev-lang/python-2.7* =dev-lang/python-2.6* )" which is very similar
> to the way that QT3VERSIONS works in qt3.eclass. One thing that is
> sub-optimal about these types of dependencies is that you end up
> with lots of installed packages that have out-dated dependencies
> when the next minor version of python is released (python-2.8 in
> this case). In the case of the python dependencies, it might be more
> optimal to use a version range like ">=dev-lang/python-2.6
> <dev-lang/python-3".

Thing is, the first deps are valid- the deps you posted however 
aren't and cannot be used as you're proposing.

Under || ( dev-lang/python:2.7 dev-lang/python:2.6 )
Having python:2.6 or python:2.7 merged satisfies it.

Under >=dev-lang/python:2.6 <dev-lang/python:3.0
having "|| ( python:2.6 python:2.7 )" satisfies it, as does 
"|| ( python:2.4 python:2.5 ) || ( python:3.0 python:3.1 python:3.2 )"

Literally, python:2.5 and python:3.1 merged would satisfy it, which is 
completely contrary to the intent and an unlikely scenario (several of 
my machines have such a deployment).

Ranged versions don't exist in any EAPI's currently although you *can* 
get away with them in the cases where the target has a single 
slotting- only in that case will the ranged versions from above work.  
Python obviously has multiple slottings, thus you cannot rely on it.

If you want ranged versions, get them added to an EAPI- but do *not* 
introduce deps like you proposed into the tree since they're wrong 
(related, kindly fix the portage deps since I pointed this issue out 
several months back).

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 1533 bytes --]

2010-02-08 01:20:22 Brian Harring napisał(a):
> On Sun, Feb 07, 2010 at 12:17:17PM -0800, Zac Medico wrote:
> > I noticed that this generates a depedency like "|| (
> > =dev-lang/python-2.7* =dev-lang/python-2.6* )" which is very similar
> > to the way that QT3VERSIONS works in qt3.eclass. One thing that is
> > sub-optimal about these types of dependencies is that you end up
> > with lots of installed packages that have out-dated dependencies
> > when the next minor version of python is released (python-2.8 in
> > this case). In the case of the python dependencies, it might be more
> > optimal to use a version range like ">=dev-lang/python-2.6
> > <dev-lang/python-3".
> 
> Thing is, the first deps are valid- the deps you posted however 
> aren't and cannot be used as you're proposing.
> 
> Under || ( dev-lang/python:2.7 dev-lang/python:2.6 )
> Having python:2.6 or python:2.7 merged satisfies it.
> 
> Under >=dev-lang/python:2.6 <dev-lang/python:3.0
> having "|| ( python:2.6 python:2.7 )" satisfies it, as does 
> "|| ( python:2.4 python:2.5 ) || ( python:3.0 python:3.1 python:3.2 )"
> 
> Literally, python:2.5 and python:3.1 merged would satisfy it, which is 
> completely contrary to the intent and an unlikely scenario (several of 
> my machines have such a deployment).

I will improve handling of cases when minimal and maximal versions aren't
specified. E.g. PYTHON_DEPEND="2" will be translated into dependency on
"=dev-lang/python-2*".

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 2399 bytes --]

Arfrever Frehtes Taifersar Arahesis
2010-02-05 17:40:00  napisał(a):
> The following changes should be made in ebuilds of packages supporting only Python 2:
> - Dependency on Python 2 should be set correctly. You can specify it directly in
>   {,R}DEPEND or use PYTHON_DEPEND.
> 
>     Example:
>       PYTHON_DEPEND="2"
> 
>       inherit python
> 
> - Miscellaneous actions should be executed with correct version of Python.
> 
>   - In ebuilds of packages supporting installation for multiple versions of Python,
>     please remember to disable installation for all versions of Python 3.
>     (Please do not use python_set_active_version() in ebuilds of these packages.)
> 
>     Example:
>       RESTRICT_PYTHON_ABIS="3.*"
> 
>   - In ebuilds of packages not supporting installation for multiple versions of Python,
>     set active version of Python using python_set_active_version().
>     (Please avoid passing precisely specified Python ABI to python_set_active_version().)
> 
>     Example:
>       pkg_setup() {
>           python_set_active_version 2
>       }
> 
>     (If given package currently cannot be installed for multiple versions of Python,
>     but it is theoretically possible (e.g. this package installs .py files into
>     /usr/lib/pythonX.Y/site-packages and does not install any non-versioned libraries
>     into /usr/lib), then please try to adjust ebuilds of this package to allow
>     installation for multiple versions of Python.)

I forgot about 2 other types of changes:
- Shebangs in installed scripts should be correct. If build system installs scripts
  with too generic shebangs (e.g. '#!/usr/bin/python'), then you can use
  python_convert_shebangs() to convert shebangs.

  Example:
    src_install() {
        ...

        python_convert_shebangs -r 2 "${D}usr/bin"
    }

  (Please note that wrapper scripts generated by python_generate_wrapper_scripts() work
  with all versions of Python from 2.4 to 3.2, so shebangs in these scripts do not need
  any changes.)

- Some deprecated functions (python_version(), distutils_python_version()) and variables
  (${python}, ${PYVER} etc.) should not be used.

Documentation [1] has been updated.

I will start filing bugs maybe in this week.

[1] http://www.gentoo.org/proj/en/Python/developersguide.xml

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Python-3.2-related changes

[Max Arnold]

On Mon, Mar 01, 2010 at 04:13:10AM +0100, Arfrever Frehtes Taifersar Arahesis wrote:
>   (Please note that wrapper scripts generated by python_generate_wrapper_scripts() work
>   with all versions of Python from 2.4 to 3.2, so shebangs in these scripts do not need
>   any changes.)

What is the recommended policy about using or not using wrapper scripts? Maybe it also should
be documented?

> [1] http://www.gentoo.org/proj/en/Python/developersguide.xml

What about merging (or at least linking) this documentation with Gentoo Development Guide? The
latter one already contains distutils related chapter and probably it is a good idea to decribe
python.eclass usage in another chapter or eclass reference.

Re: [gentoo-dev] Python-3.2-related changes

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 1136 bytes --]

2010-03-01 06:13:19 Max Arnold napisał(a):
> On Mon, Mar 01, 2010 at 04:13:10AM +0100, Arfrever Frehtes Taifersar Arahesis wrote:
> >   (Please note that wrapper scripts generated by python_generate_wrapper_scripts() work
> >   with all versions of Python from 2.4 to 3.2, so shebangs in these scripts do not need
> >   any changes.)
> 
> What is the recommended policy about using or not using wrapper scripts?

They are to solve collisions between scripts installed with different Python versions,
not to support Python 3.

> Maybe it also should be documented?

I will document it.

> > [1] http://www.gentoo.org/proj/en/Python/developersguide.xml
> 
> What about merging (or at least linking) this documentation with Gentoo Development Guide? The
> latter one already contains distutils related chapter and probably it is a good idea to decribe
> python.eclass usage in another chapter or eclass reference.

It's better to maintain our documentation separately. Gentoo Development Guide can
contain URL to http://www.gentoo.org/proj/en/Python/developersguide.xml.

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]