* [gentoo-dev] Please port your packages to Python 3.8 @ 2020-09-01 11:02 Michał Górny 2020-09-01 12:06 ` Rich Freeman 2020-09-02 17:23 ` Sam James 0 siblings, 2 replies; 20+ messages in thread From: Michał Górny @ 2020-09-01 11:02 UTC (permalink / raw To: gentoo-dev-announce; +Cc: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1152 bytes --] Hello, Following the timeline published earlier (and copied to [1]), Python 3.7 deprecation starts today. All developers are requested to install Python 3.8 and start testing their packages against it. While at it, testing against Python 3.9 would also be welcome. QA reports provide a list [2] and a graph [3] of packages needing porting. To be honest, the only major nuisance in Python 3.8 I can recall right now is that 'python-3.8' pkg-config file no longer includes '-lpython3.8' and packages that need it need to switch to 'python-3.8-embed' instead. If you can think of other problems, please ping me and I'll add some tips to python-guide [4] soonish. As usual, please add or fix tests to your packages. Python 3.8 is planned to become the default on 2020-12-01. Thank you all for your hard work! [1] https://wiki.gentoo.org/wiki/Project:Python/Implementations#Implementation_support_timeline [2] https://qa-reports.gentoo.org/output/gpyutils/37-to-38.txt [3] https://qa-reports.gentoo.org/output/gpyutils/37-to-38.svg [4] https://dev.gentoo.org/~mgorny/python-guide/ -- Best regards, Michał Górny [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 618 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-01 11:02 [gentoo-dev] Please port your packages to Python 3.8 Michał Górny @ 2020-09-01 12:06 ` Rich Freeman 2020-09-01 12:59 ` Michał Górny ` (2 more replies) 2020-09-02 17:23 ` Sam James 1 sibling, 3 replies; 20+ messages in thread From: Rich Freeman @ 2020-09-01 12:06 UTC (permalink / raw To: gentoo-dev On Tue, Sep 1, 2020 at 7:02 AM Michał Górny <mgorny@gentoo.org> wrote: > > QA reports provide a list [2] and a graph [3] of packages needing > porting. These lists would be far more useful if they actually listed the maintainer(s) of each package. Then devs could just grep to discover if any of their packages need fixing. Otherwise I fear that you're just going to run into the same issue as last time - most devs will just wait until you take the time to do this and file bugs. -- Rich ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-01 12:06 ` Rich Freeman @ 2020-09-01 12:59 ` Michał Górny 2020-09-01 13:09 ` Alexey Sokolov 2020-09-02 7:40 ` Michał Górny 2 siblings, 0 replies; 20+ messages in thread From: Michał Górny @ 2020-09-01 12:59 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 658 bytes --] On Tue, 2020-09-01 at 08:06 -0400, Rich Freeman wrote: > On Tue, Sep 1, 2020 at 7:02 AM Michał Górny <mgorny@gentoo.org> wrote: > > QA reports provide a list [2] and a graph [3] of packages needing > > porting. > > These lists would be far more useful if they actually listed the > maintainer(s) of each package. Then devs could just grep to discover > if any of their packages need fixing. > > Otherwise I fear that you're just going to run into the same issue as > last time - most devs will just wait until you take the time to do > this and file bugs. Hmm, I suppose this won't be hard to add. -- Best regards, Michał Górny [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 618 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-01 12:06 ` Rich Freeman 2020-09-01 12:59 ` Michał Górny @ 2020-09-01 13:09 ` Alexey Sokolov 2020-09-02 7:40 ` Michał Górny 2 siblings, 0 replies; 20+ messages in thread From: Alexey Sokolov @ 2020-09-01 13:09 UTC (permalink / raw To: gentoo-dev вт, 1 сент. 2020 г. в 13:06, Rich Freeman <rich0@gentoo.org>: > > On Tue, Sep 1, 2020 at 7:02 AM Michał Górny <mgorny@gentoo.org> wrote: > > > > QA reports provide a list [2] and a graph [3] of packages needing > > porting. > > These lists would be far more useful if they actually listed the > maintainer(s) of each package. Then devs could just grep to discover > if any of their packages need fixing. > > Otherwise I fear that you're just going to run into the same issue as > last time - most devs will just wait until you take the time to do > this and file bugs. > I would also like a warning shown in p.g.o if the package is in one of these qa lists ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-01 12:06 ` Rich Freeman 2020-09-01 12:59 ` Michał Górny 2020-09-01 13:09 ` Alexey Sokolov @ 2020-09-02 7:40 ` Michał Górny 2 siblings, 0 replies; 20+ messages in thread From: Michał Górny @ 2020-09-02 7:40 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 665 bytes --] On Tue, 2020-09-01 at 08:06 -0400, Rich Freeman wrote: > On Tue, Sep 1, 2020 at 7:02 AM Michał Górny <mgorny@gentoo.org> wrote: > > QA reports provide a list [2] and a graph [3] of packages needing > > porting. > > These lists would be far more useful if they actually listed the > maintainer(s) of each package. Then devs could just grep to discover > if any of their packages need fixing. > > Otherwise I fear that you're just going to run into the same issue as > last time - most devs will just wait until you take the time to do > this and file bugs. > ...and done. Now you can grep by your name. -- Best regards, Michał Górny [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 618 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-01 11:02 [gentoo-dev] Please port your packages to Python 3.8 Michał Górny 2020-09-01 12:06 ` Rich Freeman @ 2020-09-02 17:23 ` Sam James 2020-09-02 17:42 ` Michael Orlitzky 1 sibling, 1 reply; 20+ messages in thread From: Sam James @ 2020-09-02 17:23 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1295 bytes --] > On 1 Sep 2020, at 12:02, Michał Górny <mgorny@gentoo.org> wrote: > > Hello, > > [snip] > Python 3.8 is planned to become the default on 2020-12-01. > Note that this means we need to keep in mind stabilisation of Python 3.8+ supporting-packages going forward. Please request stabilisation of your Python 3.8+ changes at the 30 days point, or earlier if it’s a trivial revbump (as new Python targets are equivalent to new USE flags, there is no real need for a revbump unless doing other tidying whilst there). Stabling Python 3.8 packages also makes things a bit easier for people who do testing of their packages on a stable system - mostly thinking of proxy-maintainers or casual user contributions here. I’m sure you’ve all already seen my annoying bugs to keep track of needed stabilisations; it will however be worth it to ensure a smooth transition when the default changes. The danger of insufficient stabled changes is users having to understand possibly complex Portage conflicts and add large numbers of package.use exceptions on a temporary basis which is unnecessary confusion. Again a reminder to add Python 3.9 whenever you can too. > Thank you all for your hard work! > -- > Best regards, > Michał Górny > Thanks, Sam [-- Attachment #2: Message signed with OpenPGP --] [-- Type: application/pgp-signature, Size: 358 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-02 17:23 ` Sam James @ 2020-09-02 17:42 ` Michael Orlitzky 2020-09-02 18:08 ` Andreas Sturmlechner 0 siblings, 1 reply; 20+ messages in thread From: Michael Orlitzky @ 2020-09-02 17:42 UTC (permalink / raw To: gentoo-dev On 2020-09-02 13:23, Sam James wrote: > > Please request stabilisation of your Python 3.8+ changes at the 30 days point, or earlier if it’s a trivial revbump > (as new Python targets are equivalent to new USE flags, there is no real need for a revbump unless doing other tidying > whilst there). > New USE flags generally change dependencies (as is the case here), so a new revision ensures that people are forced to install the ebuild that supports python-3.8. Otherwise, you will eventually find a lot of people stuck unable to upgrade because they have an ebuild installed that only supports <=python-3.7, and were never prompted to install the copy that supports python-3.8. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-02 17:42 ` Michael Orlitzky @ 2020-09-02 18:08 ` Andreas Sturmlechner 2020-09-02 19:00 ` Michael Orlitzky 0 siblings, 1 reply; 20+ messages in thread From: Andreas Sturmlechner @ 2020-09-02 18:08 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 587 bytes --] On Wednesday, 2 September 2020 19:42:33 CEST Michael Orlitzky wrote: > New USE flags generally change dependencies (as is the case here), so a > new revision ensures that people are forced to install the ebuild that > supports python-3.8. Otherwise, you will eventually find a lot of people > stuck unable to upgrade because they have an ebuild installed that only > supports <=python-3.7, and were never prompted to install the copy that > supports python-3.8. Python target changes must be done with -U, also documented by the accompanying repository news item, not really a problem. [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 618 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-02 18:08 ` Andreas Sturmlechner @ 2020-09-02 19:00 ` Michael Orlitzky 2020-09-03 16:38 ` Alexis Ballier 0 siblings, 1 reply; 20+ messages in thread From: Michael Orlitzky @ 2020-09-02 19:00 UTC (permalink / raw To: gentoo-dev; +Cc: Andreas Sturmlechner On 2020-09-02 14:08, Andreas Sturmlechner wrote: > On Wednesday, 2 September 2020 19:42:33 CEST Michael Orlitzky wrote: >> New USE flags generally change dependencies (as is the case here), so a >> new revision ensures that people are forced to install the ebuild that >> supports python-3.8. Otherwise, you will eventually find a lot of people >> stuck unable to upgrade because they have an ebuild installed that only >> supports <=python-3.7, and were never prompted to install the copy that >> supports python-3.8. > > Python target changes must be done with -U, also documented by the > accompanying repository news item, not really a problem. > If you want to write the GLEP that obsoletes the PMS, I might even support it at this point. But until then, requiring --changed-use to have a functional system is not allowed. Any PMS-compliant package manager must be able to use ::gentoo, including one that does not implement portage-only heuristics. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-02 19:00 ` Michael Orlitzky @ 2020-09-03 16:38 ` Alexis Ballier 2020-09-03 18:17 ` Michael Orlitzky 0 siblings, 1 reply; 20+ messages in thread From: Alexis Ballier @ 2020-09-03 16:38 UTC (permalink / raw To: gentoo-dev On Wed, 2 Sep 2020 15:00:27 -0400 Michael Orlitzky <mjo@gentoo.org> wrote: > On 2020-09-02 14:08, Andreas Sturmlechner wrote: > > On Wednesday, 2 September 2020 19:42:33 CEST Michael Orlitzky wrote: > >> New USE flags generally change dependencies (as is the case here), > >> so a new revision ensures that people are forced to install the > >> ebuild that supports python-3.8. Otherwise, you will eventually > >> find a lot of people stuck unable to upgrade because they have an > >> ebuild installed that only supports <=python-3.7, and were never > >> prompted to install the copy that supports python-3.8. > > > > Python target changes must be done with -U, also documented by the > > accompanying repository news item, not really a problem. > > > > If you want to write the GLEP that obsoletes the PMS, I might even > support it at this point. But until then, requiring --changed-use to > have a functional system is not allowed. Any PMS-compliant package > manager must be able to use ::gentoo, including one that does not > implement portage-only heuristics. > ? if some upgrade wants a package with unmatched deps (e.g. not installed at all or py38 usedep not satisfied), $PM will surely try to satisfy it by installing an ebuild. I don't think PMS specifies this, nor should it. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-03 16:38 ` Alexis Ballier @ 2020-09-03 18:17 ` Michael Orlitzky 2020-09-04 8:39 ` [gentoo-dev] " Martin Vaeth 2020-09-04 12:54 ` [gentoo-dev] " Alexis Ballier 0 siblings, 2 replies; 20+ messages in thread From: Michael Orlitzky @ 2020-09-03 18:17 UTC (permalink / raw To: gentoo-dev On 2020-09-03 12:38, Alexis Ballier wrote: > > if some upgrade wants a package with unmatched deps (e.g. not installed > at all or py38 usedep not satisfied), $PM will surely try to satisfy > it by installing an ebuild. I don't think PMS specifies this, nor should > it. > It's not an upgrade per se. The situation is roughly this: 1. User installs foo-1.2.3.ebuild, which supports python-3.7. 2. Developer ninja-edits the ebuild to support python-3.8. 3a. (crickets) 4a. Developer removes python-3.7 support from foo-1.2.3.ebuild. 5a. The next time something pulls foo-1.2.3 into the depgraph, the PM will see that the installed version of foo-1.2.3 depends on python-3.7, but that there is no python-3.7 in the tree or that it's masked. Now emerge always fails. or.. 3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support. 4b. A user tries to install that revdep, but the PM sees that the latest version of foo is already installed, and it (the installed version) doesn't support python-3.8. Mysterious error messages and manual intervention ensue. What's happening is that the PM is using the metadata from the installed version of the package, rather than the ninja-edited metadata in the repo (how would it know which ebuilds were edited meaningfully?). That's completely legal according to the PMS, and also the smart thing to do: sourcing a few thousand lines of bash *just in case* there was an important change in some ebuild is a huge waste of users' time. Developers have an easy way to signal that there was an important change: to bump the "r" number at the end of the ebuild. This forces any upgrade involving e.g. foo-1.2.3 to pull in foo-1.2.3-r1, and the fact that an upgrade is available is evident from `ls`, rather than from sourcing a mountain of bash. One developer makes a change, and it saves thousands of users each a lot of time. That's what we're all here for. A package manager that uses the installed metadata is acting within its rights (both pkgcore and portage without dynamic deps do it), so we shouldn't be doing anything to break them in ::gentoo. Requiring --changed-use is both insisting that users' waste time finding out something that the developer could have told them, and also precluding the use of a package manager that doesn't implement the (unspecified) --changed-use flag in the same way portage does. tl;dr the name of an ebuild is the only sensible (and PMS-compliant) way to determine if something important changed inside it. The PMS says that a new revision will be noticed by the PM, and it doesn't specify any heuristics like --changed-use that ask the PM to (slowly) guess at it. So, we should use revisions for these things. ^ permalink raw reply [flat|nested] 20+ messages in thread
* [gentoo-dev] Re: Please port your packages to Python 3.8 2020-09-03 18:17 ` Michael Orlitzky @ 2020-09-04 8:39 ` Martin Vaeth 2020-09-04 12:40 ` Michael Orlitzky 2020-09-04 12:54 ` [gentoo-dev] " Alexis Ballier 1 sibling, 1 reply; 20+ messages in thread From: Martin Vaeth @ 2020-09-04 8:39 UTC (permalink / raw To: gentoo-dev Michael Orlitzky <mjo@gentoo.org> wrote: > What's happening is that the PM is using the metadata from the installed > version of the package, rather than the ninja-edited metadata in the > repo (how would it know which ebuilds were edited meaningfully?). The question is easy to answer: It is reasonable to assume that the repository metadata is correct and more up-to-date than the metadata of the installed version. But the PM would need to be smart enough to interpret subslot-deps correctly and merge that from the data of installed packages in a clever way. > That's completely legal according to the PMS, and also the > smart thing to do: s/smart/dumb/, but necessary for a dumb PM. > sourcing a few thousand lines of bash *just in case* there was an > important change in some ebuild is a huge waste of users' time. That's wrong: The metadata has to be (re)generated anyway, independent of whether this is a revision bump or just a modification. For most syncing methods the bash sourcing does not happen on the user's machine, and on those machines where the metadata is actually calculated, there are checksums and timestamps used to minimize the effort. From this perspective there is no difference between a modification or a revision bump. > Developers have an easy way to signal that there was an important > change: to bump the "r" number at the end of the ebuild. That's why it was decided to do it this way instead of requiring a smarter PM. Instead of putting some smart logic into the PM (which in case of portage was not fully implemented and in case of other package managers not at all), the decision was to force recompilations for every user and every tiny dependency change. > One developer makes a change, and it saves > thousands of users each a lot of time. ... costs thousands of users a lot of compilation time, although the recompilation is completely pointless for each of them, but done only to keep the PM simpler. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Re: Please port your packages to Python 3.8 2020-09-04 8:39 ` [gentoo-dev] " Martin Vaeth @ 2020-09-04 12:40 ` Michael Orlitzky 2020-09-04 18:27 ` Martin Vaeth 0 siblings, 1 reply; 20+ messages in thread From: Michael Orlitzky @ 2020-09-04 12:40 UTC (permalink / raw To: gentoo-dev On 2020-09-04 04:39, Martin Vaeth wrote: > >> That's completely legal according to the PMS, and also the >> smart thing to do: > > s/smart/dumb/, but necessary for a dumb PM Word games notwithstanding, these are the package managers described by the PMS. >> sourcing a few thousand lines of bash *just in case* there was an >> important change in some ebuild is a huge waste of users' time. > > That's wrong: The metadata has to be (re)generated anyway, > independent of whether this is a revision bump or just a modification. > For most syncing methods the bash sourcing does not happen on the > user's machine, and on those machines where the metadata is actually > calculated, there are checksums and timestamps used to minimize the > effort. From this perspective there is no difference between a > modification or a revision bump. This is Hollywood accounting =) Who generates the metadata when I `git pull`? Or in overlays? It's me, but if you record that wasted time in a different "metadata generation" ledger, then it looks like I've saved a bunch of time because the "bash sourcing" ledger reads zero. Even if I believe in a metadata angel and if we pretend that the PMS requires the metadata to be there, then rebuilding whenever metadata changes is still not 100% correct (as you point out), because it often rebuilds pointlessly. But that's getting into a harder problem. > ... costs thousands of users a lot of compilation time, although > the recompilation is completely pointless for each of them, but > done only to keep the PM simpler. The recompilation isn't always pointless. In the present scenario, the rebuild is what installs the python-3.8 copy of the program. I'm not arguing that this is the best way to do things, but that it's the only way to do them given what's in the PMS today. The foundation is always looking for ways to spend money; maybe it should pay a few people to sit around and think about this for a week. In any case, until a better solution finds its way into the PMS, we should make the revisions. ^ permalink raw reply [flat|nested] 20+ messages in thread
* [gentoo-dev] Re: Please port your packages to Python 3.8 2020-09-04 12:40 ` Michael Orlitzky @ 2020-09-04 18:27 ` Martin Vaeth 2020-09-05 9:38 ` Martin Vaeth 0 siblings, 1 reply; 20+ messages in thread From: Martin Vaeth @ 2020-09-04 18:27 UTC (permalink / raw To: gentoo-dev Michael Orlitzky <mjo@gentoo.org> wrote: > > Who generates the metadata when I `git pull`? For the gentoo repository, it is in general some gentoo server which then pushes the calculated metadata to the repository which you pull as a user. It is *possible* to use the "plain" repository, but you have to set up quite a bit for that (updating metadata is only one of several steps which you have to do manually in that case). A collection of scripts which do the missing steps is https://github.com/vaeth/portage-postsyncd-mv though I do not know whether they still work. Indeed (as probably most users) I am using since years one of the repositories with generated metadata already. > Or in overlays? There are overlays which provide metadata, and there are overlays where you have to do it on your own with egencache. Overlays which use eclasses from the gentoo repository usually do the latter, since otherwise the metadata is soon out-of-date. However, for most overlays, egencache takes far less than a minute, so for overlays, this is not really an issue. For the gentoo repository, the time of a full metadata generation is considerable. As mentioned, checksums and timestamps are used to minimize the amount, though this does not work for eclass changes (see below). > but if you record that wasted time in a different "metadata generation" > ledger, then it looks like I've saved a bunch of time because the "bash > sourcing" ledger reads zero. No. You need correct metadata after every syncing of the repository. Either the gentoo server or your machine has to do it. This is independent of whether the PM "prefers" the installed or the repositories' metadata. Excluding installed packages from metadata updating would not be sane and would not safe much time (since the vast majority of ebuilds are not installed, anyway). > Even if I believe in a metadata angel and if we pretend that the PMS > requires the metadata to be there, then rebuilding whenever metadata > changes is still not 100% correct (as you point out), because it often > rebuilds pointlessly. But that's getting into a harder problem. Yes, usually the metadata rebuilds due to eclass changes are pointless (except in the few cases where the eclass change is related with the metadata). I remember when I used to sync from the "plain" repository and rebuilt the metadata on my system, that the syncing (i.e. metadata regeneration) costed 10-30 minutes whenever one of the basic eclasses (which are sourced by almost every ebuild) had a trivial change. Probably, meanwhile machines are slightly faster and there are less such "basic" eclasses needed in newer EAPIs, but it will still need a considerable time. > The recompilation isn't always pointless. In the present scenario, the > rebuild is what installs the python-3.8 copy of the program. No. If users use the defaults for PYTHON_{,SINGLE}_TARGET, the rebuild has absolutely no effect except for changing some metadata in /var/db/pkg (and some file timestamps). > I'm not arguing that this is the best way to do things, but that it's > the only way to do them given what's in the PMS today. Of course. That was the decision made some years ago. > maybe it should pay a few people to sit around and think about > this for a week. There was such a debate some years ago, before the decision mentioned above was made. It was a hefty discussion, but there were strong proponents of pointless recompilation vs. improvement of the dependency handling (for both sides, there are much more arguments which I will not repeat now). The discussion might have turned if I would have found the time to implement the necessary change in portage, but neither then nor now I have the time to do so. (To be honest: Maybe I had the time, but I dislike python too much.) Nevertheless, I do not think that it was a good decision. I am not posting this to re-roll the above discussion again. But your posting shows that apparently not everybody knows the reasons why things are the way they are now. ^ permalink raw reply [flat|nested] 20+ messages in thread
* [gentoo-dev] Re: Please port your packages to Python 3.8 2020-09-04 18:27 ` Martin Vaeth @ 2020-09-05 9:38 ` Martin Vaeth 0 siblings, 0 replies; 20+ messages in thread From: Martin Vaeth @ 2020-09-05 9:38 UTC (permalink / raw To: gentoo-dev Martin Vaeth <martin@mvath.de> wrote: > >> Even if I believe in a metadata angel and if we pretend that the PMS >> requires the metadata to be there, then rebuilding whenever metadata >> changes is still not 100% correct (as you point out), because it often >> rebuilds pointlessly. But that's getting into a harder problem. Oh, I think I misunderstood you here. If the PM would always "prefer" the repository's metadata (if available) over the installed metadata, it would not be necessary to rebuild packages only because the metadata has changed (or, alternatively, portage could just update the installed metadata in such cases). A "forced" rebuild would then only be necessary in special situations, e.g. if a subslot dependency resolves differently. That's why prefering repository metadata over installed metadata requires some "smartness" of the package manager; there are still several corner cases where it is a political decision whether to rebuild. Currently, portage has this smartness only partially (subslot resolving does not work), and portage has no mechanism to just update installed metadata without recompilation. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-03 18:17 ` Michael Orlitzky 2020-09-04 8:39 ` [gentoo-dev] " Martin Vaeth @ 2020-09-04 12:54 ` Alexis Ballier 2020-09-04 13:06 ` Michael Orlitzky 1 sibling, 1 reply; 20+ messages in thread From: Alexis Ballier @ 2020-09-04 12:54 UTC (permalink / raw To: gentoo-dev On Thu, 3 Sep 2020 14:17:06 -0400 Michael Orlitzky <mjo@gentoo.org> wrote: > On 2020-09-03 12:38, Alexis Ballier wrote: > > > > if some upgrade wants a package with unmatched deps (e.g. not > > installed at all or py38 usedep not satisfied), $PM will surely try > > to satisfy it by installing an ebuild. I don't think PMS specifies > > this, nor should it. > > > > It's not an upgrade per se. The situation is roughly this: > > 1. User installs foo-1.2.3.ebuild, which supports python-3.7. > 2. Developer ninja-edits the ebuild to support python-3.8. > 3a. (crickets) > 4a. Developer removes python-3.7 support from foo-1.2.3.ebuild. > 5a. The next time something pulls foo-1.2.3 into the depgraph, > the PM will see that the installed version of foo-1.2.3 depends > on python-3.7, but that there is no python-3.7 in the tree or that > it's masked. Now emerge always fails. py37 will (*) still be installed as it cannot be depcleaned because of 1. emerge won't fail since deps are satisfied. (*) or rather should, but I think the only case that matters is a valid system state where noone forced uninstall of a needed package or manually rm'ed some random files > > or.. > > 3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support. > 4b. A user tries to install that revdep, but the PM sees that > the latest version of foo is already installed, and it (the > installed version) doesn't support python-3.8. Mysterious > error messages and manual intervention ensue. precisely the case I wrote above: unsatisfied dep -> pull ebuild. non-issue too. > What's happening is that the PM is using the metadata from the > installed version of the package, rather than the ninja-edited > metadata in the repo (how would it know which ebuilds were edited > meaningfully?). That's completely legal according to the PMS, and > also the smart thing to do: sourcing a few thousand lines of bash > *just in case* there was an important change in some ebuild is a huge > waste of users' time. you seem to be confusing dynamic deps and unsatisfied deps here. A package installed with py38 disabled (e.g. after a revbump) or no py38 support at all (e.g. without revbump) will not satisfy a [py38] usedep in any case so will work just the same > Developers have an easy way to signal that there was an important > change: to bump the "r" number at the end of the ebuild. This forces > any upgrade involving e.g. foo-1.2.3 to pull in foo-1.2.3-r1, and the > fact that an upgrade is available is evident from `ls`, rather than > from sourcing a mountain of bash. One developer makes a change, and > it saves thousands of users each a lot of time. That's what we're all > here for. fixing non-issues does not seem so important to me :/ [...] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-04 12:54 ` [gentoo-dev] " Alexis Ballier @ 2020-09-04 13:06 ` Michael Orlitzky 2020-09-04 13:13 ` Alexis Ballier 2020-09-04 13:22 ` Rich Freeman 0 siblings, 2 replies; 20+ messages in thread From: Michael Orlitzky @ 2020-09-04 13:06 UTC (permalink / raw To: gentoo-dev On 2020-09-04 08:54, Alexis Ballier wrote: > > py37 will (*) still be installed as it cannot be depcleaned because of > 1. emerge won't fail since deps are satisfied. > > > (*) or rather should, but I think the only case that matters is a valid > system state where noone forced uninstall of a needed package or > manually rm'ed some random files > There's no need to speculate; use pkgcore for a month and come back and tell me how much comfort these hypotheticals were. >> or.. >> >> 3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support. >> 4b. A user tries to install that revdep, but the PM sees that >> the latest version of foo is already installed, and it (the >> installed version) doesn't support python-3.8. Mysterious >> error messages and manual intervention ensue. > > > precisely the case I wrote above: unsatisfied dep -> pull ebuild. > non-issue too. It's easy to say "well this is not an issue because it can be solved by <thing no package manager does and is not part of the PMS>..." If it's easy, get it added to the PMS and I'll agree with you. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-04 13:06 ` Michael Orlitzky @ 2020-09-04 13:13 ` Alexis Ballier 2020-09-04 13:22 ` Rich Freeman 1 sibling, 0 replies; 20+ messages in thread From: Alexis Ballier @ 2020-09-04 13:13 UTC (permalink / raw To: gentoo-dev On Fri, 4 Sep 2020 09:06:46 -0400 Michael Orlitzky <mjo@gentoo.org> wrote: > On 2020-09-04 08:54, Alexis Ballier wrote: > > > > py37 will (*) still be installed as it cannot be depcleaned because > > of 1. emerge won't fail since deps are satisfied. > > > > > > (*) or rather should, but I think the only case that matters is a > > valid system state where noone forced uninstall of a needed package > > or manually rm'ed some random files > > > > There's no need to speculate; use pkgcore for a month and come back > and tell me how much comfort these hypotheticals were. there's no speculation in assuming a consistent set of installed packages (consistent as specified in... PMS ;) ); there is, however, speculation in the hypothetical error messages when the installed set of packages is inconsistent > >> or.. > >> > >> 3b. Some reverse dependency of foo-1.2.3 gets python-3.8 support. > >> 4b. A user tries to install that revdep, but the PM sees that > >> the latest version of foo is already installed, and it (the > >> installed version) doesn't support python-3.8. Mysterious > >> error messages and manual intervention ensue. > > > > > > precisely the case I wrote above: unsatisfied dep -> pull ebuild. > > non-issue too. > > It's easy to say "well this is not an issue because it can be solved > by <thing no package manager does and is not part of the PMS>..." are you kidding ? are you seriously suggesting adding to PMS that PM needs to pull ebuilds to install packages ? good luck with that ;) ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-04 13:06 ` Michael Orlitzky 2020-09-04 13:13 ` Alexis Ballier @ 2020-09-04 13:22 ` Rich Freeman 2020-09-04 13:40 ` Michael Orlitzky 1 sibling, 1 reply; 20+ messages in thread From: Rich Freeman @ 2020-09-04 13:22 UTC (permalink / raw To: gentoo-dev On Fri, Sep 4, 2020 at 9:06 AM Michael Orlitzky <mjo@gentoo.org> wrote: > > It's easy to say "well this is not an issue because it can be solved by > <thing no package manager does and is not part of the PMS>..." > > If it's easy, get it added to the PMS and I'll agree with you. > Current Gentoo policy: "Maintainers must not assume that dynamic dependencies will be applied by the package manager. When changing runtime dependencies the maintainer should revision the ebuild if the changes are likely to cause problems for end users." [1] Certainly having a discussion about whether this could change down the road is reasonable, but keep in mind this would require package managers to actually be changed, which requires code. Out of the box portage has issues with dynamic deps[2] so it isn't a solved problem on any package manager, let alone all of them. In the interim, devs MUST revbump in these situations. The Council left some room for discretion, and as a result I end up having portage rebuild everything on changed deps because frankly I don't trust people to get it right, since if people can't see for themselves the reason for a rule it seems to be a reason to ignore it. The rule is also documented in the devmanual[3]. 1 - https://projects.gentoo.org/council/meeting-logs/20151011-summary.txt 2 - https://wiki.gentoo.org/wiki/Project:Portage/Changed_Deps#Ebuild_Revision_Bumps 3 - https://devmanual.gentoo.org/general-concepts/ebuild-revisions/index.html -- Rich ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-dev] Please port your packages to Python 3.8 2020-09-04 13:22 ` Rich Freeman @ 2020-09-04 13:40 ` Michael Orlitzky 0 siblings, 0 replies; 20+ messages in thread From: Michael Orlitzky @ 2020-09-04 13:40 UTC (permalink / raw To: gentoo-dev On 2020-09-04 09:22, Rich Freeman wrote: > > Current Gentoo policy: > > ...if the changes are likely to cause problems for end users." If you're willing to ignore the user reports of problems, and ignore the mailing list threads telling you that it will cause problems, and avoid running any tests, then it becomes possible to construct a fortress of flawed reasoning to defend the claim that "this won't cause problems for end users." ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2020-09-05 9:38 UTC | newest] Thread overview: 20+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-09-01 11:02 [gentoo-dev] Please port your packages to Python 3.8 Michał Górny 2020-09-01 12:06 ` Rich Freeman 2020-09-01 12:59 ` Michał Górny 2020-09-01 13:09 ` Alexey Sokolov 2020-09-02 7:40 ` Michał Górny 2020-09-02 17:23 ` Sam James 2020-09-02 17:42 ` Michael Orlitzky 2020-09-02 18:08 ` Andreas Sturmlechner 2020-09-02 19:00 ` Michael Orlitzky 2020-09-03 16:38 ` Alexis Ballier 2020-09-03 18:17 ` Michael Orlitzky 2020-09-04 8:39 ` [gentoo-dev] " Martin Vaeth 2020-09-04 12:40 ` Michael Orlitzky 2020-09-04 18:27 ` Martin Vaeth 2020-09-05 9:38 ` Martin Vaeth 2020-09-04 12:54 ` [gentoo-dev] " Alexis Ballier 2020-09-04 13:06 ` Michael Orlitzky 2020-09-04 13:13 ` Alexis Ballier 2020-09-04 13:22 ` Rich Freeman 2020-09-04 13:40 ` Michael Orlitzky
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox