[gentoo-commits] proj/portage:prefix commit in: lib/portage/

[gentoo-commits] proj/portage:prefix commit in: lib/portage/

[Fabian Groffen]

commit:     de2a3ae590e743504e4c1c262eb2a92a70efac57
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Jun  4 09:16:30 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue Jun  4 09:16:30 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=de2a3ae5

portage.const_autotool: drop unnecessary import

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 lib/portage/const_autotool.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/lib/portage/const_autotool.py b/lib/portage/const_autotool.py
index d12003c46..ee540319c 100644
--- a/lib/portage/const_autotool.py
+++ b/lib/portage/const_autotool.py
@@ -7,8 +7,6 @@ __all__ = ["EPREFIX", "SYSCONFDIR", "PORTAGE_BASE",
 		"portageuser", "portagegroup", "rootuser", "rootuid", "rootgid",
 		"PORTAGE_BASH", "PORTAGE_MV"]
 
-from os import path
-
 EPREFIX      = "@PORTAGE_EPREFIX@"
 SYSCONFDIR   = "@sysconfdir@"
 PORTAGE_BASE = "@PORTAGE_BASE@"

[gentoo-commits] proj/portage:prefix commit in: lib/portage/

[Fabian Groffen]

commit:     c7198b5b3eb17e8b20930dc83b3a210f09802a1d
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Dec  4 04:59:49 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Jan  4 10:41:54 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=c7198b5b

_get_lock_fn: support multiprocessing spawn start method (bug 758230)

Ensure that _get_lock_fn arguments to multiprocessing.Process will
successfully pickle, as required by the spawn start method, which
is the default for macOS since Python 3.8.

Since file descriptors are not inherited unless the fork start
method is used, the subprocess should only try to close an
inherited file descriptor for the fork start method.

Bug: https://bugs.gentoo.org/758230
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 lib/portage/locks.py | 36 +++++++++++++++++++++++-------------
 1 file changed, 23 insertions(+), 13 deletions(-)

diff --git a/lib/portage/locks.py b/lib/portage/locks.py
index 1073343be..193045c03 100644
--- a/lib/portage/locks.py
+++ b/lib/portage/locks.py
@@ -44,18 +44,7 @@ def _get_lock_fn():
 	if _lock_fn is not None:
 		return _lock_fn
 
-	def _test_lock(fd, lock_path):
-		os.close(fd)
-		try:
-			with open(lock_path, 'a') as f:
-				fcntl.lockf(f.fileno(), fcntl.LOCK_EX|fcntl.LOCK_NB)
-		except EnvironmentError as e:
-			if e.errno == errno.EAGAIN:
-				# Parent process holds lock, as expected.
-				sys.exit(0)
 
-		# Something went wrong.
-		sys.exit(1)
 
 	fd, lock_path = tempfile.mkstemp()
 	try:
@@ -64,8 +53,16 @@ def _get_lock_fn():
 		except EnvironmentError:
 			pass
 		else:
-			proc = multiprocessing.Process(target=_test_lock,
-				args=(fd, lock_path))
+			proc = multiprocessing.Process(
+				target=_subprocess_test_lock,
+				args=(
+					# Since file descriptors are not inherited unless the fork start
+					# method is used, the subprocess should only try to close an
+					# inherited file descriptor for the fork start method.
+					fd if multiprocessing.get_start_method() == "fork" else None,
+					lock_path,
+				),
+			)
 			proc.start()
 			proc.join()
 			if proc.exitcode == os.EX_OK:
@@ -80,6 +77,19 @@ def _get_lock_fn():
 	_lock_fn = fcntl.flock
 	return _lock_fn
 
+def _subprocess_test_lock(fd, lock_path):
+	if fd is not None:
+		os.close(fd)
+	try:
+		with open(lock_path, 'a') as f:
+			fcntl.lockf(f.fileno(), fcntl.LOCK_EX|fcntl.LOCK_NB)
+	except EnvironmentError as e:
+		if e.errno == errno.EAGAIN:
+			# Parent process holds lock, as expected.
+			sys.exit(0)
+
+	# Something went wrong.
+	sys.exit(1)
 
 _open_fds = {}
 _open_inodes = {}

[gentoo-commits] proj/portage:prefix commit in: lib/portage/

[Fabian Groffen]

commit:     13b2f0d1edd4f70c3bf776503cdc505ccf04f2d3
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Jan  4 12:06:25 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Jan  4 12:06:25 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=13b2f0d1

lib/portage: add more definite fix for macOS multiprocessing default

force 'fork' method for as long as we get pickle errors whilst using
'spawn'

Bug: https://bugs.gentoo.org/758230
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 lib/portage/__init__.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/lib/portage/__init__.py b/lib/portage/__init__.py
index a5cc8fb08..ec1b50107 100644
--- a/lib/portage/__init__.py
+++ b/lib/portage/__init__.py
@@ -19,6 +19,8 @@ try:
 	import re
 	import types
 	import platform
+	# PREFIX LOCAL
+	import multiprocessing
 
 	# Temporarily delete these imports, to ensure that only the
 	# wrapped versions are imported by portage internals.
@@ -38,6 +40,16 @@ except ImportError as e:
 	sys.stderr.write("    "+str(e)+"\n\n")
 	raise
 
+# BEGIN PREFIX LOCAL
+# for bug #758230, on macOS the default was switched from fork to spawn,
+# the latter causing issues because all kinds of things can't be
+# pickled, so force fork mode for now
+try:
+	multiprocessing.set_start_method('fork')
+except RuntimeError:
+	pass
+# END PREFIX LOCAL
+
 try:
 
 	import portage.proxy.lazyimport

[gentoo-commits] proj/portage:prefix commit in: lib/portage/

[Fabian Groffen]

commit:     3feffbd536f3578c5ba03245ba33ae7876b60d9d
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jul 24 09:14:47 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jul 24 09:14:47 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=3feffbd5

lib/portage/data: fix undefined ref after merge

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 lib/portage/data.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/portage/data.py b/lib/portage/data.py
index 0dac72845..6848751fe 100644
--- a/lib/portage/data.py
+++ b/lib/portage/data.py
@@ -177,6 +177,7 @@ def _get_global(k):
             portage_uid = pwd.getpwnam(_get_global("_portage_username")).pw_uid
         except KeyError:
             # PREFIX LOCAL: some sysadmins are insane, bug #344307
+            username = _get_global("_portage_username")
             if username.isdigit():
                 portage_uid = int(username)
             else:
@@ -188,6 +189,7 @@ def _get_global(k):
             portage_gid = grp.getgrnam(_get_global("_portage_grpname")).gr_gid
         except KeyError:
             # PREFIX LOCAL: some sysadmins are insane, bug #344307
+            grpname = _get_global("_portage_grpname")
             if grpname.isdigit():
                 portage_gid = int(grpname)
             else:

[gentoo-commits] proj/portage:prefix commit in: lib/portage/

[Fabian Groffen]

commit:     89e49c9e04d878f627e9f88eebac5124d649db3c
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Nov 24 19:28:01 2023 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Nov 24 20:05:51 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=89e49c9e

lib/portage: drop multiprocessing fork force

After Zac's efforts we should be able to use spawn now.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 lib/portage/__init__.py | 10 ----------
 1 file changed, 10 deletions(-)

diff --git a/lib/portage/__init__.py b/lib/portage/__init__.py
index 4cd465f9e5..2886bd29cc 100644
--- a/lib/portage/__init__.py
+++ b/lib/portage/__init__.py
@@ -53,16 +53,6 @@ except ImportError as e:
     sys.stderr.write(f"    {e}\n\n")
     raise
 
-# BEGIN PREFIX LOCAL
-# for bug #758230, on macOS the default was switched from fork to spawn,
-# the latter causing issues because all kinds of things can't be
-# pickled, so force fork mode for now
-try:
-    multiprocessing.set_start_method('fork')
-except RuntimeError:
-    pass
-# END PREFIX LOCAL
-
 try:
     import portage.proxy.lazyimport
     import portage.proxy as proxy

[gentoo-commits] proj/portage:prefix commit in: lib/portage/

[Fabian Groffen]

commit:     82ebd8e271b0c17e279ef02b1f247cf6fa298a60
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Nov 24 20:53:05 2023 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Nov 24 20:53:05 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=82ebd8e2

meson: install const_autotool.py on Prefix

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 lib/portage/meson.build | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/portage/meson.build b/lib/portage/meson.build
index 06dde8ca71..6a648fd9d0 100644
--- a/lib/portage/meson.build
+++ b/lib/portage/meson.build
@@ -21,6 +21,7 @@ py.install_sources(
         'binpkg.py',
         'checksum.py',
         const_py,
+        'const_autotool.py',  # PREFIX_LOCAL
         'cvstree.py',
         'data.py',
         'debug.py',

[gentoo-commits] proj/portage:prefix commit in: lib/portage/

[Fabian Groffen]

commit:     b83451f801b314e3022bb0ca6b5d88e03b0bc5d5
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Jan 18 10:38:10 2024 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Jan 18 10:38:10 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=b83451f8

lib/portage: drop obsolete multiprocessing import

This is a leftover from the macOS multiprocessing fork force we had to
do in the past.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 lib/portage/__init__.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/lib/portage/__init__.py b/lib/portage/__init__.py
index 2886bd29cc..f097b653c7 100644
--- a/lib/portage/__init__.py
+++ b/lib/portage/__init__.py
@@ -12,8 +12,6 @@ try:
     import asyncio
     import sys
     import errno
-    # PREFIX LOCAL
-    import multiprocessing
 
     if not hasattr(errno, "ESTALE"):
         # ESTALE may not be defined on some systems, such as interix.