[gentoo-dev] Please port your packages to Python 3.8

[gentoo-dev] Please port your packages to Python 3.8

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1152 bytes --]

Hello,

Following the timeline published earlier (and copied to [1]), Python 3.7
deprecation starts today.  All developers are requested to install
Python 3.8 and start testing their packages against it.  While at it,
testing against Python 3.9 would also be welcome.

QA reports provide a list [2] and a graph [3] of packages needing
porting.  To be honest, the only major nuisance in Python 3.8 I can
recall right now is that 'python-3.8' pkg-config file no longer includes
'-lpython3.8' and packages that need it need to switch to
'python-3.8-embed' instead.  If you can think of other problems, please
ping me and I'll add some tips to python-guide [4] soonish.

As usual, please add or fix tests to your packages.

Python 3.8 is planned to become the default on 2020-12-01.

Thank you all for your hard work!


[1] https://wiki.gentoo.org/wiki/Project:Python/Implementations#Implementation_support_timeline
[2] https://qa-reports.gentoo.org/output/gpyutils/37-to-38.txt
[3] https://qa-reports.gentoo.org/output/gpyutils/37-to-38.svg
[4] https://dev.gentoo.org/~mgorny/python-guide/

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Please port your packages to Python 3.8

[Rich Freeman]

On Tue, Sep 1, 2020 at 7:02 AM Michał Górny <mgorny@gentoo.org> wrote:
>
> QA reports provide a list [2] and a graph [3] of packages needing
> porting.

These lists would be far more useful if they actually listed the
maintainer(s) of each package.  Then devs could just grep to discover
if any of their packages need fixing.

Otherwise I fear that you're just going to run into the same issue as
last time - most devs will just wait until you take the time to do
this and file bugs.

-- 
Rich

Re: [gentoo-dev] Please port your packages to Python 3.8

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

On Tue, 2020-09-01 at 08:06 -0400, Rich Freeman wrote:
> On Tue, Sep 1, 2020 at 7:02 AM Michał Górny <mgorny@gentoo.org> wrote:
> > QA reports provide a list [2] and a graph [3] of packages needing
> > porting.
> 
> These lists would be far more useful if they actually listed the
> maintainer(s) of each package.  Then devs could just grep to discover
> if any of their packages need fixing.
> 
> Otherwise I fear that you're just going to run into the same issue as
> last time - most devs will just wait until you take the time to do
> this and file bugs.

Hmm, I suppose this won't be hard to add.

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Please port your packages to Python 3.8

[Alexey Sokolov]

вт, 1 сент. 2020 г. в 13:06, Rich Freeman <rich0@gentoo.org>:
>
> On Tue, Sep 1, 2020 at 7:02 AM Michał Górny <mgorny@gentoo.org> wrote:
> >
> > QA reports provide a list [2] and a graph [3] of packages needing
> > porting.
>
> These lists would be far more useful if they actually listed the
> maintainer(s) of each package.  Then devs could just grep to discover
> if any of their packages need fixing.
>
> Otherwise I fear that you're just going to run into the same issue as
> last time - most devs will just wait until you take the time to do
> this and file bugs.
>

I would also like a warning shown in p.g.o if the package is in one of
these qa lists

Re: [gentoo-dev] Please port your packages to Python 3.8

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]

On Tue, 2020-09-01 at 08:06 -0400, Rich Freeman wrote:
> On Tue, Sep 1, 2020 at 7:02 AM Michał Górny <mgorny@gentoo.org> wrote:
> > QA reports provide a list [2] and a graph [3] of packages needing
> > porting.
> 
> These lists would be far more useful if they actually listed the
> maintainer(s) of each package.  Then devs could just grep to discover
> if any of their packages need fixing.
> 
> Otherwise I fear that you're just going to run into the same issue as
> last time - most devs will just wait until you take the time to do
> this and file bugs.
> 

...and done.  Now you can grep by your name.

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Please port your packages to Python 3.8

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1295 bytes --]


> On 1 Sep 2020, at 12:02, Michał Górny <mgorny@gentoo.org> wrote:
> 
> Hello,
> 
> [snip]

> Python 3.8 is planned to become the default on 2020-12-01.
> 

Note that this means we need to keep in mind stabilisation of Python 3.8+ supporting-packages going forward.

Please request stabilisation of your Python 3.8+ changes at the 30 days point, or earlier if it’s a trivial revbump
(as new Python targets are equivalent to new USE flags, there is no real need for a revbump unless doing other tidying
whilst there).

Stabling Python 3.8 packages also makes things a bit easier for people who do testing of their packages on a stable system -
mostly thinking of proxy-maintainers or casual user contributions here.

I’m sure you’ve all already seen my annoying bugs to keep track of needed stabilisations; it will however be worth
it to ensure a smooth transition when the default changes.

The danger of insufficient stabled changes is users having to understand possibly complex Portage conflicts and add
large numbers of package.use exceptions on a temporary basis which is unnecessary confusion.

Again a reminder to add Python 3.9 whenever you can too.

> Thank you all for your hard work!
> --
> Best regards,
> Michał Górny
> 

Thanks,
Sam


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: [gentoo-dev] Please port your packages to Python 3.8

[Michael Orlitzky]

On 2020-09-02 13:23, Sam James wrote:
> 
> Please request stabilisation of your Python 3.8+ changes at the 30 days point, or earlier if it’s a trivial revbump
> (as new Python targets are equivalent to new USE flags, there is no real need for a revbump unless doing other tidying
> whilst there).
> 

New USE flags generally change dependencies (as is the case here), so a
new revision ensures that people are forced to install the ebuild that
supports python-3.8. Otherwise, you will eventually find a lot of people
stuck unable to upgrade because they have an ebuild installed that only
supports <=python-3.7, and were never prompted to install the copy that
supports python-3.8.

Re: [gentoo-dev] Please port your packages to Python 3.8

[Andreas Sturmlechner]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Wednesday, 2 September 2020 19:42:33 CEST Michael Orlitzky wrote:
> New USE flags generally change dependencies (as is the case here), so a
> new revision ensures that people are forced to install the ebuild that
> supports python-3.8. Otherwise, you will eventually find a lot of people
> stuck unable to upgrade because they have an ebuild installed that only
> supports <=python-3.7, and were never prompted to install the copy that
> supports python-3.8.

Python target changes must be done with -U, also documented by the 
accompanying repository news item, not really a problem.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Please port your packages to Python 3.8

[Michael Orlitzky]

On 2020-09-02 14:08, Andreas Sturmlechner wrote:
> On Wednesday, 2 September 2020 19:42:33 CEST Michael Orlitzky wrote:
>> New USE flags generally change dependencies (as is the case here), so a
>> new revision ensures that people are forced to install the ebuild that
>> supports python-3.8. Otherwise, you will eventually find a lot of people
>> stuck unable to upgrade because they have an ebuild installed that only
>> supports <=python-3.7, and were never prompted to install the copy that
>> supports python-3.8.
> 
> Python target changes must be done with -U, also documented by the 
> accompanying repository news item, not really a problem.
> 

If you want to write the GLEP that obsoletes the PMS, I might even
support it at this point. But until then, requiring --changed-use to
have a functional system is not allowed. Any PMS-compliant package
manager must be able to use ::gentoo, including one that does not
implement portage-only heuristics.

Re: [gentoo-dev] Please port your packages to Python 3.8

[Alexis Ballier]

On Wed, 2 Sep 2020 15:00:27 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 2020-09-02 14:08, Andreas Sturmlechner wrote:
> > On Wednesday, 2 September 2020 19:42:33 CEST Michael Orlitzky wrote:
> >> New USE flags generally change dependencies (as is the case here),
> >> so a new revision ensures that people are forced to install the
> >> ebuild that supports python-3.8. Otherwise, you will eventually
> >> find a lot of people stuck unable to upgrade because they have an
> >> ebuild installed that only supports <=python-3.7, and were never
> >> prompted to install the copy that supports python-3.8.
> > 
> > Python target changes must be done with -U, also documented by the 
> > accompanying repository news item, not really a problem.
> > 
> 
> If you want to write the GLEP that obsoletes the PMS, I might even
> support it at this point. But until then, requiring --changed-use to
> have a functional system is not allowed. Any PMS-compliant package
> manager must be able to use ::gentoo, including one that does not
> implement portage-only heuristics.
> 

?

if some upgrade wants a package with unmatched deps (e.g. not installed
at all or py38 usedep not satisfied), $PM will surely try to satisfy
it by installing an ebuild. I don't think PMS specifies this, nor should
it.

Re: [gentoo-dev] Please port your packages to Python 3.8

[Michael Orlitzky]

On 2020-09-03 12:38, Alexis Ballier wrote:
> 
> if some upgrade wants a package with unmatched deps (e.g. not installed
> at all or py38 usedep not satisfied), $PM will surely try to satisfy
> it by installing an ebuild. I don't think PMS specifies this, nor should
> it.
> 

It's not an upgrade per se. The situation is roughly this:

  1. User installs foo-1.2.3.ebuild, which supports python-3.7.
  2. Developer ninja-edits the ebuild to support python-3.8.
  3a. (crickets)
  4a. Developer removes python-3.7 support from foo-1.2.3.ebuild.
  5a. The next time something pulls foo-1.2.3 into the depgraph,
      the PM will see that the installed version of foo-1.2.3 depends on
      python-3.7, but that there is no python-3.7 in the tree or that
      it's masked. Now emerge always fails.

or..

  3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support.
  4b. A user tries to install that revdep, but the PM sees that
      the latest version of foo is already installed, and it (the
      installed version) doesn't support python-3.8. Mysterious
      error messages and manual intervention ensue.

What's happening is that the PM is using the metadata from the installed
version of the package, rather than the ninja-edited metadata in the
repo (how would it know which ebuilds were edited meaningfully?). That's
completely legal according to the PMS, and also the smart thing to do:
sourcing a few thousand lines of bash *just in case* there was an
important change in some ebuild is a huge waste of users' time.

Developers have an easy way to signal that there was an important
change: to bump the "r" number at the end of the ebuild. This forces any
upgrade involving e.g. foo-1.2.3 to pull in foo-1.2.3-r1, and the fact
that an upgrade is available is evident from `ls`, rather than from
sourcing a mountain of bash. One developer makes a change, and it saves
thousands of users each a lot of time. That's what we're all here for.

A package manager that uses the installed metadata is acting within its
rights (both pkgcore and portage without dynamic deps do it), so we
shouldn't be doing anything to break them in ::gentoo. Requiring
--changed-use is both insisting that users' waste time finding out
something that the developer could have told them, and also precluding
the use of a package manager that doesn't implement the (unspecified)
--changed-use flag in the same way portage does.

tl;dr the name of an ebuild is the only sensible (and PMS-compliant) way
to determine if something important changed inside it. The PMS says that
a new revision will be noticed by the PM, and it doesn't specify any
heuristics like --changed-use that ask the PM to (slowly) guess at it.
So, we should use revisions for these things.

[gentoo-dev] Re: Please port your packages to Python 3.8

[Martin Vaeth]

Michael Orlitzky <mjo@gentoo.org> wrote:
> What's happening is that the PM is using the metadata from the installed
> version of the package, rather than the ninja-edited metadata in the
> repo (how would it know which ebuilds were edited meaningfully?).

The question is easy to answer:
It is reasonable to assume that the repository metadata is correct
and more up-to-date than the metadata of the installed version.
But the PM would need to be smart enough to interpret subslot-deps
correctly and merge that from the data of installed packages in a
clever way.

> That's completely legal according to the PMS, and also the
> smart thing to do:

s/smart/dumb/, but necessary for a dumb PM.

> sourcing a few thousand lines of bash *just in case* there was an
> important change in some ebuild is a huge waste of users' time.

That's wrong: The metadata has to be (re)generated anyway,
independent of whether this is a revision bump or just a modification.
For most syncing methods the bash sourcing does not happen on the
user's machine, and on those machines where the metadata is actually
calculated, there are checksums and timestamps used to minimize the
effort. From this perspective there is no difference between a
modification or a revision bump.

> Developers have an easy way to signal that there was an important
> change: to bump the "r" number at the end of the ebuild.

That's why it was decided to do it this way instead of requiring
a smarter PM. Instead of putting some smart logic into the PM
(which in case of portage was not fully implemented and in case
of other package managers not at all), the decision was to force
recompilations for every user and every tiny dependency change.

> One developer makes a change, and it saves
> thousands of users each a lot of time.

... costs thousands of users a lot of compilation time, although
the recompilation is completely pointless for each of them, but
done only to keep the PM simpler.

Re: [gentoo-dev] Re: Please port your packages to Python 3.8

[Michael Orlitzky]

On 2020-09-04 04:39, Martin Vaeth wrote:
> 
>> That's completely legal according to the PMS, and also the
>> smart thing to do:
> 
> s/smart/dumb/, but necessary for a dumb PM
Word games notwithstanding, these are the package managers described by
the PMS.


>> sourcing a few thousand lines of bash *just in case* there was an
>> important change in some ebuild is a huge waste of users' time.
> 
> That's wrong: The metadata has to be (re)generated anyway,
> independent of whether this is a revision bump or just a modification.
> For most syncing methods the bash sourcing does not happen on the
> user's machine, and on those machines where the metadata is actually
> calculated, there are checksums and timestamps used to minimize the
> effort. From this perspective there is no difference between a
> modification or a revision bump.

This is Hollywood accounting =)

Who generates the metadata when I `git pull`? Or in overlays? It's me,
but if you record that wasted time in a different "metadata generation"
ledger, then it looks like I've saved a bunch of time because the "bash
sourcing" ledger reads zero.

Even if I believe in a metadata angel and if we pretend that the PMS
requires the metadata to be there, then rebuilding whenever metadata
changes is still not 100% correct (as you point out), because it often
rebuilds pointlessly. But that's getting into a harder problem.


> ... costs thousands of users a lot of compilation time, although
> the recompilation is completely pointless for each of them, but
> done only to keep the PM simpler.

The recompilation isn't always pointless. In the present scenario, the
rebuild is what installs the python-3.8 copy of the program.

I'm not arguing that this is the best way to do things, but that it's
the only way to do them given what's in the PMS today. The foundation is
always looking for ways to spend money; maybe it should pay a few people
to sit around and think about this for a week. In any case, until a
better solution finds its way into the PMS, we should make the revisions.

Re: [gentoo-dev] Please port your packages to Python 3.8

[Alexis Ballier]

On Thu, 3 Sep 2020 14:17:06 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 2020-09-03 12:38, Alexis Ballier wrote:
> > 
> > if some upgrade wants a package with unmatched deps (e.g. not
> > installed at all or py38 usedep not satisfied), $PM will surely try
> > to satisfy it by installing an ebuild. I don't think PMS specifies
> > this, nor should it.
> > 
> 
> It's not an upgrade per se. The situation is roughly this:
> 
>   1. User installs foo-1.2.3.ebuild, which supports python-3.7.
>   2. Developer ninja-edits the ebuild to support python-3.8.
>   3a. (crickets)
>   4a. Developer removes python-3.7 support from foo-1.2.3.ebuild.
>   5a. The next time something pulls foo-1.2.3 into the depgraph,
>       the PM will see that the installed version of foo-1.2.3 depends
> on python-3.7, but that there is no python-3.7 in the tree or that
>       it's masked. Now emerge always fails.


py37 will (*) still be installed as it cannot be depcleaned because of
1. emerge won't fail since deps are satisfied.


(*) or rather should, but I think the only case that matters is a valid
system state where noone forced uninstall of a needed package or
manually rm'ed some random files

> 
> or..
> 
>   3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support.
>   4b. A user tries to install that revdep, but the PM sees that
>       the latest version of foo is already installed, and it (the
>       installed version) doesn't support python-3.8. Mysterious
>       error messages and manual intervention ensue.


precisely the case I wrote above: unsatisfied dep -> pull ebuild.
non-issue too.


> What's happening is that the PM is using the metadata from the
> installed version of the package, rather than the ninja-edited
> metadata in the repo (how would it know which ebuilds were edited
> meaningfully?). That's completely legal according to the PMS, and
> also the smart thing to do: sourcing a few thousand lines of bash
> *just in case* there was an important change in some ebuild is a huge
> waste of users' time.

you seem to be confusing dynamic deps and unsatisfied deps here. A
package installed with py38 disabled (e.g. after a revbump) or no py38
support at all (e.g. without revbump) will not satisfy a [py38] usedep
in any case so will work just the same


> Developers have an easy way to signal that there was an important
> change: to bump the "r" number at the end of the ebuild. This forces
> any upgrade involving e.g. foo-1.2.3 to pull in foo-1.2.3-r1, and the
> fact that an upgrade is available is evident from `ls`, rather than
> from sourcing a mountain of bash. One developer makes a change, and
> it saves thousands of users each a lot of time. That's what we're all
> here for.

fixing non-issues does not seem so important to me :/

[...]

Re: [gentoo-dev] Please port your packages to Python 3.8

[Michael Orlitzky]

On 2020-09-04 08:54, Alexis Ballier wrote:
> 
> py37 will (*) still be installed as it cannot be depcleaned because of
> 1. emerge won't fail since deps are satisfied.
> 
> 
> (*) or rather should, but I think the only case that matters is a valid
> system state where noone forced uninstall of a needed package or
> manually rm'ed some random files
> 

There's no need to speculate; use pkgcore for a month and come back and
tell me how much comfort these hypotheticals were.


>> or..
>>
>>   3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support.
>>   4b. A user tries to install that revdep, but the PM sees that
>>       the latest version of foo is already installed, and it (the
>>       installed version) doesn't support python-3.8. Mysterious
>>       error messages and manual intervention ensue.
> 
> 
> precisely the case I wrote above: unsatisfied dep -> pull ebuild.
> non-issue too.

It's easy to say "well this is not an issue because it can be solved by
<thing no package manager does and is not part of the PMS>..."

If it's easy, get it added to the PMS and I'll agree with you.

Re: [gentoo-dev] Please port your packages to Python 3.8

[Alexis Ballier]

On Fri, 4 Sep 2020 09:06:46 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 2020-09-04 08:54, Alexis Ballier wrote:
> > 
> > py37 will (*) still be installed as it cannot be depcleaned because
> > of 1. emerge won't fail since deps are satisfied.
> > 
> > 
> > (*) or rather should, but I think the only case that matters is a
> > valid system state where noone forced uninstall of a needed package
> > or manually rm'ed some random files
> > 
> 
> There's no need to speculate; use pkgcore for a month and come back
> and tell me how much comfort these hypotheticals were.

there's no speculation in assuming a consistent set of installed
packages (consistent as specified in... PMS ;) ); there is, however,
speculation in the hypothetical error messages when the installed set
of packages is inconsistent


> >> or..
> >>
> >>   3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support.
> >>   4b. A user tries to install that revdep, but the PM sees that
> >>       the latest version of foo is already installed, and it (the
> >>       installed version) doesn't support python-3.8. Mysterious
> >>       error messages and manual intervention ensue.
> > 
> > 
> > precisely the case I wrote above: unsatisfied dep -> pull ebuild.
> > non-issue too.
> 
> It's easy to say "well this is not an issue because it can be solved
> by <thing no package manager does and is not part of the PMS>..."

are you kidding ? are you seriously suggesting adding to PMS that PM
needs to pull ebuilds to install packages ? good luck with that ;)

Re: [gentoo-dev] Please port your packages to Python 3.8

[Rich Freeman]

On Fri, Sep 4, 2020 at 9:06 AM Michael Orlitzky <mjo@gentoo.org> wrote:
>
> It's easy to say "well this is not an issue because it can be solved by
> <thing no package manager does and is not part of the PMS>..."
>
> If it's easy, get it added to the PMS and I'll agree with you.
>

Current Gentoo policy:

"Maintainers must not assume that dynamic dependencies will be applied
by the package manager. When changing runtime dependencies the
maintainer should revision the ebuild if the changes are likely to
cause problems for end users." [1]

Certainly having a discussion about whether this could change down the
road is reasonable, but keep in mind this would require package
managers to actually be changed, which requires code.

Out of the box portage has issues with dynamic deps[2] so it isn't a
solved problem on any package manager, let alone all of them.

In the interim, devs MUST revbump in these situations.  The Council
left some room for discretion, and as a result I end up having portage
rebuild everything on changed deps because frankly I don't trust
people to get it right, since if people can't see for themselves the
reason for a rule it seems to be a reason to ignore it.

The rule is also documented in the devmanual[3].

1 - https://projects.gentoo.org/council/meeting-logs/20151011-summary.txt
2 - https://wiki.gentoo.org/wiki/Project:Portage/Changed_Deps#Ebuild_Revision_Bumps
3 - https://devmanual.gentoo.org/general-concepts/ebuild-revisions/index.html

-- 
Rich

Re: [gentoo-dev] Please port your packages to Python 3.8

[Michael Orlitzky]

On 2020-09-04 09:22, Rich Freeman wrote:
> 
> Current Gentoo policy:
> 
> ...if the changes are likely to cause problems for end users." 

If you're willing to ignore the user reports of problems, and ignore the
mailing list threads telling you that it will cause problems, and avoid
running any tests, then it becomes possible to construct a fortress of
flawed reasoning to defend the claim that "this won't cause problems for
end users."

[gentoo-dev] Re: Please port your packages to Python 3.8

[Martin Vaeth]

Michael Orlitzky <mjo@gentoo.org> wrote:
>
> Who generates the metadata when I `git pull`?

For the gentoo repository, it is in general some gentoo server
which then pushes the calculated metadata to the repository which
you pull as a user.
It is *possible* to use the "plain" repository, but you have to
set up quite a bit for that (updating metadata is only one of
several steps which you have to do manually in that case).

A collection of scripts which do the missing steps is
https://github.com/vaeth/portage-postsyncd-mv
though I do not know whether they still work. Indeed (as
probably most users) I am using since years one of the
repositories with generated metadata already.

> Or in overlays?

There are overlays which provide metadata, and there are overlays
where you have to do it on your own with egencache. Overlays which
use eclasses from the gentoo repository usually do the latter,
since otherwise the metadata is soon out-of-date.

However, for most overlays, egencache takes far less than a minute,
so for overlays, this is not really an issue. For the gentoo repository,
the time of a full metadata generation is considerable. As mentioned,
checksums and timestamps are used to minimize the amount, though this
does not work for eclass changes (see below).

> but if you record that wasted time in a different "metadata generation"
> ledger, then it looks like I've saved a bunch of time because the "bash
> sourcing" ledger reads zero.

No. You need correct metadata after every syncing of the repository.
Either the gentoo server or your machine has to do it.
This is independent of whether the PM "prefers" the installed or
the repositories' metadata. Excluding installed packages from metadata
updating would not be sane and would not safe much time (since the vast
majority of ebuilds are not installed, anyway).

> Even if I believe in a metadata angel and if we pretend that the PMS
> requires the metadata to be there, then rebuilding whenever metadata
> changes is still not 100% correct (as you point out), because it often
> rebuilds pointlessly. But that's getting into a harder problem.

Yes, usually the metadata rebuilds due to eclass changes are pointless
(except in the few cases where the eclass change is related with the
metadata).

I remember when I used to sync from the "plain" repository and rebuilt
the metadata on my system, that the syncing (i.e. metadata regeneration)
costed 10-30 minutes whenever one of the basic eclasses (which are
sourced by almost every ebuild) had a trivial change. Probably,
meanwhile machines are slightly faster and there are less such "basic"
eclasses needed in newer EAPIs, but it will still need a considerable
time.

> The recompilation isn't always pointless. In the present scenario, the
> rebuild is what installs the python-3.8 copy of the program.

No. If users use the defaults for PYTHON_{,SINGLE}_TARGET, the
rebuild has absolutely no effect except for changing some metadata
in /var/db/pkg (and some file timestamps).

> I'm not arguing that this is the best way to do things, but that it's
> the only way to do them given what's in the PMS today.

Of course. That was the decision made some years ago.

> maybe it should pay a few people to sit around and think about
> this for a week.

There was such a debate some years ago, before the decision mentioned
above was made.
It was a hefty discussion, but there were strong proponents of pointless
recompilation vs. improvement of the dependency handling (for both
sides, there are much more arguments which I will not repeat now).
The discussion might have turned if I would have found the time to
implement the necessary change in portage, but neither then nor now
I have the time to do so. (To be honest: Maybe I had the time, but I
dislike python too much.) Nevertheless, I do not think that it was a
good decision.
I am not posting this to re-roll the above discussion again.
But your posting shows that apparently not everybody knows the reasons
why things are the way they are now.

[gentoo-dev] Re: Please port your packages to Python 3.8

[Martin Vaeth]

Martin Vaeth <martin@mvath.de> wrote:
>
>> Even if I believe in a metadata angel and if we pretend that the PMS
>> requires the metadata to be there, then rebuilding whenever metadata
>> changes is still not 100% correct (as you point out), because it often
>> rebuilds pointlessly. But that's getting into a harder problem.

Oh, I think I misunderstood you here.
If the PM would always "prefer" the repository's metadata (if available)
over the installed metadata, it would not be necessary to rebuild packages
only because the metadata has changed (or, alternatively, portage could
just update the installed metadata in such cases).
A "forced" rebuild would then only be necessary in special situations,
e.g. if a subslot dependency resolves differently. That's why prefering
repository metadata over installed metadata requires some "smartness"
of the package manager; there are still several corner cases where it is a
political decision whether to rebuild. Currently, portage has this
smartness only partially (subslot resolving does not work), and portage
has no mechanism to just update installed metadata without recompilation.