From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id A0320158099 for ; Tue, 28 Nov 2023 22:51:48 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id C6D5C2BC01F; Tue, 28 Nov 2023 22:51:47 +0000 (UTC) Received: from smtp.gentoo.org (woodpecker.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id AAAC02BC01F for ; Tue, 28 Nov 2023 22:51:47 +0000 (UTC) Received: from oystercatcher.gentoo.org (oystercatcher.gentoo.org [148.251.78.52]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id E18CC335CF4 for ; Tue, 28 Nov 2023 22:51:46 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id 4CCE72EC for ; Tue, 28 Nov 2023 22:51:45 +0000 (UTC) From: "Zac Medico" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Zac Medico" Message-ID: <1701211673.9b711229ef913e8772e9a3a1cce744c87ee468cf.zmedico@gentoo> Subject: [gentoo-commits] proj/portage:master commit in: lib/_emerge/ X-VCS-Repository: proj/portage X-VCS-Files: lib/_emerge/Binpkg.py lib/_emerge/CompositeTask.py lib/_emerge/EbuildBuild.py lib/_emerge/Scheduler.py X-VCS-Directories: lib/_emerge/ X-VCS-Committer: zmedico X-VCS-Committer-Name: Zac Medico X-VCS-Revision: 9b711229ef913e8772e9a3a1cce744c87ee468cf X-VCS-Branch: master Date: Tue, 28 Nov 2023 22:51:45 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-Archives-Salt: 316b9016-f11e-42a8-84d0-3b2b72b1ba26 X-Archives-Hash: 408b74cfcebf60f6ae0dd30b61e2ed10 commit: 9b711229ef913e8772e9a3a1cce744c87ee468cf Author: Zac Medico gentoo org> AuthorDate: Mon Nov 27 01:55:04 2023 +0000 Commit: Zac Medico gentoo org> CommitDate: Tue Nov 28 22:47:53 2023 +0000 URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=9b711229 Cancel prefetchers that have not started yet Cancel prefetchers that have not started yet. This will cause the parallel-fetch SequentialTaskQueue to simply discard them, and a regular fetcher will be instantiated immediately. This prevents errors when accessing the pkg_path attribute of BinpkgPrefetcher instances that have not started yet. Bug: https://bugs.gentoo.org/918636 Signed-off-by: Zac Medico gentoo.org> lib/_emerge/Binpkg.py | 6 +++++- lib/_emerge/CompositeTask.py | 8 +++++++- lib/_emerge/EbuildBuild.py | 4 ++++ lib/_emerge/Scheduler.py | 8 +++++++- 4 files changed, 23 insertions(+), 3 deletions(-) diff --git a/lib/_emerge/Binpkg.py b/lib/_emerge/Binpkg.py index fac8722064..9b1036538a 100644 --- a/lib/_emerge/Binpkg.py +++ b/lib/_emerge/Binpkg.py @@ -1,4 +1,4 @@ -# Copyright 1999-2021 Gentoo Authors +# Copyright 1999-2023 Gentoo Authors # Distributed under the terms of the GNU General Public License v2 import functools @@ -110,6 +110,10 @@ class Binpkg(CompositeTask): # use the scheduler and fetcher methods to # synchronize with the fetcher. prefetcher = self.prefetcher + if prefetcher is not None and not prefetcher.isAlive(): + # Cancel it because it hasn't started yet. + prefetcher.cancel() + self.prefetcher = prefetcher = None if prefetcher is None: pass elif prefetcher.isAlive() and prefetcher.poll() is None: diff --git a/lib/_emerge/CompositeTask.py b/lib/_emerge/CompositeTask.py index a0203e71f3..50c1b8c5f3 100644 --- a/lib/_emerge/CompositeTask.py +++ b/lib/_emerge/CompositeTask.py @@ -1,4 +1,4 @@ -# Copyright 1999-2018 Gentoo Foundation +# Copyright 1999-2023 Gentoo Authors # Distributed under the terms of the GNU General Public License v2 from _emerge.AsynchronousTask import AsynchronousTask @@ -10,6 +10,12 @@ class CompositeTask(AsynchronousTask): _TASK_QUEUED = -1 + def isAlive(self): + """ + Returns True if started and returncode has not been set. + """ + return self.returncode is None and self._current_task is not None + def _cancel(self): if self._current_task is not None: if self._current_task is self._TASK_QUEUED: diff --git a/lib/_emerge/EbuildBuild.py b/lib/_emerge/EbuildBuild.py index 079a67ffbe..81cbfdc085 100644 --- a/lib/_emerge/EbuildBuild.py +++ b/lib/_emerge/EbuildBuild.py @@ -96,6 +96,10 @@ class EbuildBuild(CompositeTask): return prefetcher = self.prefetcher + if prefetcher is not None and not prefetcher.isAlive(): + # Cancel it because it hasn't started yet. + prefetcher.cancel() + self.prefetcher = prefetcher = None if prefetcher is None: pass elif prefetcher.isAlive() and prefetcher.poll() is None: diff --git a/lib/_emerge/Scheduler.py b/lib/_emerge/Scheduler.py index e1179e0f13..66eca4c650 100644 --- a/lib/_emerge/Scheduler.py +++ b/lib/_emerge/Scheduler.py @@ -940,6 +940,10 @@ class Scheduler(PollScheduler): # is consuming time here. if bintree.isremote(x.cpv): fetcher = self._get_prefetcher(x) + if fetcher is not None and not fetcher.isAlive(): + # Cancel it because it hasn't started yet. + fetcher.cancel() + fetcher = None if fetcher is None: fetcher = BinpkgFetcher(pkg=x, scheduler=loop) fetcher.start() @@ -1963,7 +1967,9 @@ class Scheduler(PollScheduler): # CPython 2.7, so it may be possible for CPython to raise KeyError # here as well. prefetcher = None - if prefetcher is not None and not prefetcher.isAlive(): + if prefetcher is not None and ( + prefetcher.cancelled or not prefetcher.isAlive() + ): try: self._task_queues.fetch._task_queue.remove(prefetcher) except ValueError: