[gentoo-dev] Python 2 cleanup update

[gentoo-dev] Python 2 cleanup update

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3732 bytes --]

Hello, everyone.

TL;DR: we're nearing the total annihilation of Python 2 software
in Gentoo.  Most users could safely disable py2 USE flags today.
Python 2 vulns have been patched recently, the interpreter and a few
packages using Python at build time (with no deps) will stay.  Should we
change PYTHON_TARGETS now, or wait some more and just annihilate
the py2 flag from all packages?


Long version:

We're reached the point where the majority of packages relying on py2
have either been ported to py3, removed or masked for removal.
As a result, I've been able to eliminate python2_7 target from the vast
majority of dev-python/* packages.  On their next system upgrade, our
users are going to notice most of Python 2.7 modules gone from their
systems.

However, because of their reverse dependencies a few packages can't lose
their py2.7-iness, and therefore are going to block depcleaning Python
2.7 for now.  These include old versions of setuptools, numpy, pillow,
as well as all versions of cython, nose, pykerberos, pyyaml and their
dependencies.  The major blockers for them are:

- dev-lang/gdl (py entirely optional but the package itself is seriously
broken)

- dev-db/mongodb (py3 version was just stabilized, need to decide how to
clean old versions up)

- games-engines/renpy (no py3 version yet)

- media-tv/kodi (py3 version in alpha)

We plan to have these packages fixed or removed by the deadline. 


However, we already know that there are some packages that use Python 2
at build time and that will keep requiring it past the deadline.
The initial list includes:

- dev-python/pypy* (TODO: need to figure bootstrap out)

- dev-lang/spidermonkey, www-client/seamonkey, www-client/firefox...
(thank you, Mozilla)

- www-client/chromium, dev-qt/qtwebengine... (thank you, Google)

Sadly, the big corps are too busy improving their spying functionality
and creating NIH programming languages to take care of such minor
matters as cleaning up.

The general rule is that py2.7 may remain in packages that use it
at build time only (i.e. don't install anything depending on Python)
and have no dependencies on Python packages (i.e. don't require any
other packages to install py2.7 modules).  Or to put it otherwise,
python-r1 and python-single-r1 will lose py2.7 support entirely, while
python-any-r1 will retain minimal support without dependencies.


This also implies that we're going to keep Python 2.7 itself for as long
as necessary, and patch it if possible.  I should take this opportunity
to remind you that it's quite possible that the interpreter itself has
unknown vulnerabilities.  Only recently I've backported two sec fixes
from Python 3 which no other distribution (including the one promising
paid support for Python 2 for next years) or upstream (including all
these boasting that they're going to maintain Python 2 themselves) has
even noticed (to the best of my knowledge).


An open question is whether we should remove python2_7 from
PYTHON_TARGETS now.  If we do that, it will permit the vast majority of
Gentoo users to depclean Python 2.7 today, independently of how long
the maintainer of renpy is going to block it, with only a few users
having to enable the flag manually.  However, doing this makes sense
only if we're really going to delay the impeding doom long.

I will probably prepare an updated news item for Python 2.7 removal,
to replace the one from February with the updated plan, current
information and helpful tips.


Finally, I would like to thank all the helpful package maintainers, arch
teams and other developers who have made this possible.

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Python 2 cleanup update

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 4331 bytes --]

On Sun, Sep 27, 2020 at 10:45 AM Michał Górny <mgorny@gentoo.org> wrote:

> Hello, everyone.
>
> TL;DR: we're nearing the total annihilation of Python 2 software
> in Gentoo.  Most users could safely disable py2 USE flags today.
> Python 2 vulns have been patched recently, the interpreter and a few
> packages using Python at build time (with no deps) will stay.  Should we
> change PYTHON_TARGETS now, or wait some more and just annihilate
> the py2 flag from all packages?
>
>
> Long version:
>
> We're reached the point where the majority of packages relying on py2
> have either been ported to py3, removed or masked for removal.
> As a result, I've been able to eliminate python2_7 target from the vast
> majority of dev-python/* packages.  On their next system upgrade, our
> users are going to notice most of Python 2.7 modules gone from their
> systems.
>
> However, because of their reverse dependencies a few packages can't lose
> their py2.7-iness, and therefore are going to block depcleaning Python
> 2.7 for now.  These include old versions of setuptools, numpy, pillow,
> as well as all versions of cython, nose, pykerberos, pyyaml and their
> dependencies.  The major blockers for them are:
>
> - dev-lang/gdl (py entirely optional but the package itself is seriously
> broken)
>
> - dev-db/mongodb (py3 version was just stabilized, need to decide how to
> clean old versions up)
>
> - games-engines/renpy (no py3 version yet)
>
> - media-tv/kodi (py3 version in alpha)
>
> We plan to have these packages fixed or removed by the deadline.
>
>
> However, we already know that there are some packages that use Python 2
> at build time and that will keep requiring it past the deadline.
> The initial list includes:
>
> - dev-python/pypy* (TODO: need to figure bootstrap out)
>
> - dev-lang/spidermonkey, www-client/seamonkey, www-client/firefox...
> (thank you, Mozilla)
>
> - www-client/chromium, dev-qt/qtwebengine... (thank you, Google)
>
> Sadly, the big corps are too busy improving their spying functionality
> and creating NIH programming languages to take care of such minor
> matters as cleaning up.
>


https://bugs.chromium.org/p/chromium/issues/list?q=Proj%3DPython3Migration&can=2
is the tracker for python3 migration for chromium. I resent the implication
that Google is 'too busy' to work on it.

E.g. on
https://bugs.chromium.org/p/chromium/issues/detail?id=941669&q=Proj%3DPython3Migration&can=2
the last commit was Sept 26, or yesterday ;p

-A


> The general rule is that py2.7 may remain in packages that use it
> at build time only (i.e. don't install anything depending on Python)
> and have no dependencies on Python packages (i.e. don't require any
> other packages to install py2.7 modules).  Or to put it otherwise,
> python-r1 and python-single-r1 will lose py2.7 support entirely, while
> python-any-r1 will retain minimal support without dependencies.
>
>
> This also implies that we're going to keep Python 2.7 itself for as long
> as necessary, and patch it if possible.  I should take this opportunity
> to remind you that it's quite possible that the interpreter itself has
> unknown vulnerabilities.  Only recently I've backported two sec fixes
> from Python 3 which no other distribution (including the one promising
> paid support for Python 2 for next years) or upstream (including all
> these boasting that they're going to maintain Python 2 themselves) has
> even noticed (to the best of my knowledge).
>
>
> An open question is whether we should remove python2_7 from
> PYTHON_TARGETS now.  If we do that, it will permit the vast majority of
> Gentoo users to depclean Python 2.7 today, independently of how long
> the maintainer of renpy is going to block it, with only a few users
> having to enable the flag manually.  However, doing this makes sense
> only if we're really going to delay the impeding doom long.
>
> I will probably prepare an updated news item for Python 2.7 removal,
> to replace the one from February with the updated plan, current
> information and helpful tips.
>
>
> Finally, I would like to thank all the helpful package maintainers, arch
> teams and other developers who have made this possible.
>
> --
> Best regards,
> Michał Górny
>
>

[-- Attachment #2: Type: text/html, Size: 5350 bytes --]

Re: [gentoo-dev] Python 2 cleanup update

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2830 bytes --]

On Sun, 2020-09-27 at 11:08 -0700, Alec Warner wrote:
> On Sun, Sep 27, 2020 at 10:45 AM Michał Górny <mgorny@gentoo.org> wrote:
> 
> > Hello, everyone.
> > 
> > TL;DR: we're nearing the total annihilation of Python 2 software
> > in Gentoo.  Most users could safely disable py2 USE flags today.
> > Python 2 vulns have been patched recently, the interpreter and a few
> > packages using Python at build time (with no deps) will stay.  Should we
> > change PYTHON_TARGETS now, or wait some more and just annihilate
> > the py2 flag from all packages?
> > 
> > 
> > Long version:
> > 
> > We're reached the point where the majority of packages relying on py2
> > have either been ported to py3, removed or masked for removal.
> > As a result, I've been able to eliminate python2_7 target from the vast
> > majority of dev-python/* packages.  On their next system upgrade, our
> > users are going to notice most of Python 2.7 modules gone from their
> > systems.
> > 
> > However, because of their reverse dependencies a few packages can't lose
> > their py2.7-iness, and therefore are going to block depcleaning Python
> > 2.7 for now.  These include old versions of setuptools, numpy, pillow,
> > as well as all versions of cython, nose, pykerberos, pyyaml and their
> > dependencies.  The major blockers for them are:
> > 
> > - dev-lang/gdl (py entirely optional but the package itself is seriously
> > broken)
> > 
> > - dev-db/mongodb (py3 version was just stabilized, need to decide how to
> > clean old versions up)
> > 
> > - games-engines/renpy (no py3 version yet)
> > 
> > - media-tv/kodi (py3 version in alpha)
> > 
> > We plan to have these packages fixed or removed by the deadline.
> > 
> > 
> > However, we already know that there are some packages that use Python 2
> > at build time and that will keep requiring it past the deadline.
> > The initial list includes:
> > 
> > - dev-python/pypy* (TODO: need to figure bootstrap out)
> > 
> > - dev-lang/spidermonkey, www-client/seamonkey, www-client/firefox...
> > (thank you, Mozilla)
> > 
> > - www-client/chromium, dev-qt/qtwebengine... (thank you, Google)
> > 
> > Sadly, the big corps are too busy improving their spying functionality
> > and creating NIH programming languages to take care of such minor
> > matters as cleaning up.
> > 
> 
> https://bugs.chromium.org/p/chromium/issues/list?q=Proj%3DPython3Migration&can=2
> is the tracker for python3 migration for chromium. I resent the implication
> that Google is 'too busy' to work on it.
> 
> E.g. on
> https://bugs.chromium.org/p/chromium/issues/detail?id=941669&q=Proj%3DPython3Migration&can=2
> the last commit was Sept 26, or yesterday ;p
> 

Do you happen to have some 'starting date' too?

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Python 2 cleanup update

[Lars Wendler]

[-- Attachment #1: Type: text/plain, Size: 4510 bytes --]

Hi folks,

On Sun, 27 Sep 2020 19:45:22 +0200 Michał Górny wrote:

>Hello, everyone.
>
>TL;DR: we're nearing the total annihilation of Python 2 software
>in Gentoo.  Most users could safely disable py2 USE flags today.
>Python 2 vulns have been patched recently, the interpreter and a few
>packages using Python at build time (with no deps) will stay.  Should
>we change PYTHON_TARGETS now, or wait some more and just annihilate
>the py2 flag from all packages?
>
>
>Long version:
>
>We're reached the point where the majority of packages relying on py2
>have either been ported to py3, removed or masked for removal.
>As a result, I've been able to eliminate python2_7 target from the vast
>majority of dev-python/* packages.  On their next system upgrade, our
>users are going to notice most of Python 2.7 modules gone from their
>systems.
>
>However, because of their reverse dependencies a few packages can't
>lose their py2.7-iness, and therefore are going to block depcleaning
>Python 2.7 for now.  These include old versions of setuptools, numpy,
>pillow, as well as all versions of cython, nose, pykerberos, pyyaml
>and their dependencies.  The major blockers for them are:
>
>- dev-lang/gdl (py entirely optional but the package itself is
>seriously broken)
>
>- dev-db/mongodb (py3 version was just stabilized, need to decide how
>to clean old versions up)
>
>- games-engines/renpy (no py3 version yet)
>
>- media-tv/kodi (py3 version in alpha)
>
>We plan to have these packages fixed or removed by the deadline. 
>
>
>However, we already know that there are some packages that use Python 2
>at build time and that will keep requiring it past the deadline.
>The initial list includes:
>
>- dev-python/pypy* (TODO: need to figure bootstrap out)
>
>- dev-lang/spidermonkey, www-client/seamonkey, www-client/firefox...
>(thank you, Mozilla)

I've already talked to seamonkey upstream about this and I was told
that they will shift to python3 with seamonkey-2.57 release (which will
be the followup release to the 2.53.x series) but they could not tell
me even an approximate release date.
seamonkey upstream only has loose bindings to Mozilla (they still use
Mozilla's bugzilla but their development repos are now on gitlab) and
their man-power is quite low so I do not expect 2.57 releases before
the year 2021. I hope we can keep dev-lang/python:2.7 for the time
being.

>- www-client/chromium, dev-qt/qtwebengine... (thank you, Google)
>
>Sadly, the big corps are too busy improving their spying functionality
>and creating NIH programming languages to take care of such minor
>matters as cleaning up.
>
>The general rule is that py2.7 may remain in packages that use it
>at build time only (i.e. don't install anything depending on Python)
>and have no dependencies on Python packages (i.e. don't require any
>other packages to install py2.7 modules).  Or to put it otherwise,
>python-r1 and python-single-r1 will lose py2.7 support entirely, while
>python-any-r1 will retain minimal support without dependencies.
>
>
>This also implies that we're going to keep Python 2.7 itself for as
>long as necessary, and patch it if possible.  I should take this
>opportunity to remind you that it's quite possible that the
>interpreter itself has unknown vulnerabilities.  Only recently I've
>backported two sec fixes from Python 3 which no other distribution
>(including the one promising paid support for Python 2 for next years)
>or upstream (including all these boasting that they're going to
>maintain Python 2 themselves) has even noticed (to the best of my
>knowledge).
>
>
>An open question is whether we should remove python2_7 from
>PYTHON_TARGETS now.  If we do that, it will permit the vast majority of
>Gentoo users to depclean Python 2.7 today, independently of how long
>the maintainer of renpy is going to block it, with only a few users
>having to enable the flag manually.  However, doing this makes sense
>only if we're really going to delay the impeding doom long.
>
>I will probably prepare an updated news item for Python 2.7 removal,
>to replace the one from February with the updated plan, current
>information and helpful tips.
>
>
>Finally, I would like to thank all the helpful package maintainers,
>arch teams and other developers who have made this possible.
>


Cheers
-- 
Lars Wendler
Gentoo package maintainer
GPG: 21CC CF02 4586 0A07 ED93  9F68 498F E765 960E 9B39

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]