[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     abf6f36a2785671f183f5f898f03c29cd5a915b9
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Aug 11 19:48:01 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Aug 11 21:05:20 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=abf6f36a

Make features USE respect RESTRICT=test (bug 663278)

Make RESTRICT=test prevent the "test" USE flag from being added to
features USE flags when FEATURES=test is enabled, in order to preserve
default behavior for ebuilds that set RESTRICT=test. The code that sets
the restrict_test variable in the setcpv method must execute earlier
now, but the logic is unchanged. Note that it is still possible to
enable USE=test for ebuilds that set RESTRICT=test, but FEATURES=test
will not do it, so it will only be triggered by an explicit USE=test
setting by the user or profile.

Fixes: 8c5598c1af2c ("Replace implicit {FEATURES->USE}=test forcing with USE default")
Bug: https://bugs.gentoo.org/663278
Reviewed-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 50 ++++++++++++++++++------------------
 1 file changed, 25 insertions(+), 25 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 3e0081829..220fa31bb 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1663,7 +1663,31 @@ class config(object):
 		else:
 			iuse_implicit_match = self._iuse_implicit_match
 
-		if "test" in explicit_iuse or iuse_implicit_match("test"):
+		if pkg is None:
+			raw_restrict = pkg_configdict.get("RESTRICT")
+		else:
+			raw_restrict = pkg._raw_metadata["RESTRICT"]
+
+		restrict_test = False
+		if raw_restrict:
+			try:
+				if built_use is not None:
+					restrict = use_reduce(raw_restrict,
+						uselist=built_use, flat=True)
+				else:
+					# Use matchnone=True to ignore USE conditional parts
+					# of RESTRICT, since we want to know whether to mask
+					# the "test" flag _before_ we know the USE values
+					# that would be needed to evaluate the USE
+					# conditionals (see bug #273272).
+					restrict = use_reduce(raw_restrict,
+						matchnone=True, flat=True)
+			except PortageException:
+				pass
+			else:
+				restrict_test = "test" in restrict
+
+		if not restrict_test and ("test" in explicit_iuse or iuse_implicit_match("test")):
 			if "test" in self.features:
 				feature_use.append("test")
 
@@ -1721,30 +1745,6 @@ class config(object):
 		self.configdict["env"].addLazySingleton(
 			"PORTAGE_IUSE", _lazy_iuse_regex, portage_iuse)
 
-		if pkg is None:
-			raw_restrict = pkg_configdict.get("RESTRICT")
-		else:
-			raw_restrict = pkg._raw_metadata["RESTRICT"]
-
-		restrict_test = False
-		if raw_restrict:
-			try:
-				if built_use is not None:
-					restrict = use_reduce(raw_restrict,
-						uselist=built_use, flat=True)
-				else:
-					# Use matchnone=True to ignore USE conditional parts
-					# of RESTRICT, since we want to know whether to mask
-					# the "test" flag _before_ we know the USE values
-					# that would be needed to evaluate the USE
-					# conditionals (see bug #273272).
-					restrict = use_reduce(raw_restrict,
-						matchnone=True, flat=True)
-			except PortageException:
-				pass
-			else:
-				restrict_test = "test" in restrict
-
 		ebuild_force_test = not restrict_test and \
 			self.get("EBUILD_FORCE_TEST") == "1"
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     45986341a80cfb01dad470f56f02b210b3ebf753
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Aug 11 21:04:05 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Aug 12 00:40:32 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=45986341

Support !test? conditionals in RESTRICT (bug 663278)

Since RESTRICT="!test? ( test )" can be very useful within the context
of bug 663278, pass an appropriate uselist parameter to the RESTRICT
use_reduce call. Make self.configdict["features"]["USE"] independent
of IUSE and RESTRICT, so that the same value can be shared between
packages with different settings, which is important when evaluating
USE conditional RESTRICT. When the evaluated value of RESTRICT contains
"test", handle it like IUSE="-test", since features USE is independent
of RESTRICT.

Bug: https://bugs.gentoo.org/663278

 lib/portage/package/ebuild/config.py | 40 +++++++++++++++++++++++-------------
 1 file changed, 26 insertions(+), 14 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 220fa31bb..3b01095d0 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1457,6 +1457,7 @@ class config(object):
 		cp = cpv_getkey(mycpv)
 		cpv_slot = self.mycpv
 		pkginternaluse = ""
+		pkginternaluse_list = []
 		feature_use = []
 		iuse = ""
 		pkg_configdict = self.configdict["pkg"]
@@ -1513,13 +1514,12 @@ class config(object):
 				cpv_slot = self.mycpv
 			else:
 				cpv_slot = pkg
-			pkginternaluse = []
 			for x in iuse.split():
 				if x.startswith("+"):
-					pkginternaluse.append(x[1:])
+					pkginternaluse_list.append(x[1:])
 				elif x.startswith("-"):
-					pkginternaluse.append(x)
-			pkginternaluse = " ".join(pkginternaluse)
+					pkginternaluse_list.append(x)
+			pkginternaluse = " ".join(pkginternaluse_list)
 
 		eapi_attrs = _get_eapi_attrs(eapi)
 
@@ -1596,6 +1596,9 @@ class config(object):
 			# regenerate() call in order to ensure that self.features
 			# is accurate.
 			has_changed = True
+			# Prevent stale features USE from corrupting the evaluation
+			# of USE conditional RESTRICT.
+			self.configdict["features"]["USE"] = ""
 
 		self._penv = []
 		cpdict = self._penvdict.get(cp)
@@ -1675,24 +1678,33 @@ class config(object):
 					restrict = use_reduce(raw_restrict,
 						uselist=built_use, flat=True)
 				else:
-					# Use matchnone=True to ignore USE conditional parts
-					# of RESTRICT, since we want to know whether to mask
-					# the "test" flag _before_ we know the USE values
-					# that would be needed to evaluate the USE
-					# conditionals (see bug #273272).
 					restrict = use_reduce(raw_restrict,
-						matchnone=True, flat=True)
+						uselist=frozenset(x for x in self['USE'].split()
+						if x in explicit_iuse or iuse_implicit_match(x)),
+						flat=True)
 			except PortageException:
 				pass
 			else:
 				restrict_test = "test" in restrict
 
-		if not restrict_test and ("test" in explicit_iuse or iuse_implicit_match("test")):
-			if "test" in self.features:
-				feature_use.append("test")
+		pkginternaluse_before = pkginternaluse
+		if "test" in self.features:
+			# This is independent of IUSE and RESTRICT, so that the same
+			# value can be shared between packages with different settings,
+			# which is important when evaluating USE conditional RESTRICT
+			# above.
+			feature_use.append("test")
+
+			if restrict_test:
+				# Handle it like IUSE="-test", since features USE is
+				# independent of RESTRICT.
+				pkginternaluse_list.append("-test")
+				pkginternaluse = " ".join(pkginternaluse_list)
+				self.configdict["pkginternal"]["USE"] = pkginternaluse
 
 		feature_use = " ".join(feature_use)
-		if feature_use != self.configdict["features"].get("USE", ""):
+		if (feature_use != self.configdict["features"].get("USE", "") or
+			pkginternaluse is not pkginternaluse_before):
 			self.configdict["features"]["USE"] = feature_use
 			# TODO: can we avoid that?
 			self.reset(keeping_pkg=1)

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     ad607b8ecfad3e099cae508ff4d3b5b62cab969f
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Aug 17 11:08:21 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Aug 17 20:12:32 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=ad607b8e

config: default features USE for conditional RESTRICT (bug 663848)

Initialize default features USE state, and reset when appropriate,
so that the "test" flag state is correct for evaluation of !test?
conditionals in RESTRICT.

Fixes: 45986341a80c ("Support !test? conditionals in RESTRICT (bug 663278)")
Bug: https://bugs.gentoo.org/663848

 lib/portage/package/ebuild/config.py | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 3b01095d0..9fbf7e8fc 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -259,6 +259,7 @@ class config(object):
 			self.packages = clone.packages
 			self.repositories = clone.repositories
 			self.unpack_dependencies = clone.unpack_dependencies
+			self._default_features_use = clone._default_features_use
 			self._iuse_effective = clone._iuse_effective
 			self._iuse_implicit_match = clone._iuse_implicit_match
 			self._non_user_variables = clone._non_user_variables
@@ -961,6 +962,14 @@ class config(object):
 
 			# initialize self.features
 			self.regenerate()
+			feature_use = []
+			if "test" in self.features:
+				feature_use.append("test")
+			self.configdict["features"]["USE"] = self._default_features_use = " ".join(feature_use)
+			if feature_use:
+				# Regenerate USE so that the initial "test" flag state is
+				# correct for evaluation of !test? conditionals in RESTRICT.
+				self.regenerate()
 
 			if unprivileged:
 				self.features.add('unprivileged')
@@ -1296,7 +1305,7 @@ class config(object):
 			del self._penv[:]
 			self.configdict["pkg"].clear()
 			self.configdict["pkginternal"].clear()
-			self.configdict["features"].clear()
+			self.configdict["features"]["USE"] = self._default_features_use
 			self.configdict["repo"].clear()
 			self.configdict["defaults"]["USE"] = \
 				" ".join(self.make_defaults_use)
@@ -1598,7 +1607,7 @@ class config(object):
 			has_changed = True
 			# Prevent stale features USE from corrupting the evaluation
 			# of USE conditional RESTRICT.
-			self.configdict["features"]["USE"] = ""
+			self.configdict["features"]["USE"] = self._default_features_use
 
 		self._penv = []
 		cpdict = self._penvdict.get(cp)

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     cbe6f8cac95f5db7b35018a01c56dc3d9c76bb21
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Aug 20 21:54:14 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug 20 22:13:59 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=cbe6f8ca

config: regenerate USE for conditional RESTRICT (bug 664104)

When the features USE state changes due to package.env settings,
regenerate USE for correct evaluation of conditional RESTRICT.

Fixes: 45986341a80c ("Support !test? conditionals in RESTRICT (bug 663278)")
Bug: https://bugs.gentoo.org/664104

 lib/portage/package/ebuild/config.py | 39 ++++++++++++++++++------------------
 1 file changed, 20 insertions(+), 19 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 9fbf7e8fc..a68df5807 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1666,8 +1666,22 @@ class config(object):
 			has_changed = True
 
 		if has_changed:
+			# This can modify self.features due to package.env settings.
 			self.reset(keeping_pkg=1)
 
+		if "test" in self.features:
+			# This is independent of IUSE and RESTRICT, so that the same
+			# value can be shared between packages with different settings,
+			# which is important when evaluating USE conditional RESTRICT.
+			feature_use.append("test")
+
+		feature_use = " ".join(feature_use)
+		if feature_use != self.configdict["features"]["USE"]:
+			# Regenerate USE for evaluation of conditional RESTRICT.
+			self.configdict["features"]["USE"] = feature_use
+			self.reset(keeping_pkg=1)
+			has_changed = True
+
 		if explicit_iuse is None:
 			explicit_iuse = frozenset(x.lstrip("+-") for x in iuse.split())
 		if eapi_attrs.iuse_effective:
@@ -1696,25 +1710,12 @@ class config(object):
 			else:
 				restrict_test = "test" in restrict
 
-		pkginternaluse_before = pkginternaluse
-		if "test" in self.features:
-			# This is independent of IUSE and RESTRICT, so that the same
-			# value can be shared between packages with different settings,
-			# which is important when evaluating USE conditional RESTRICT
-			# above.
-			feature_use.append("test")
-
-			if restrict_test:
-				# Handle it like IUSE="-test", since features USE is
-				# independent of RESTRICT.
-				pkginternaluse_list.append("-test")
-				pkginternaluse = " ".join(pkginternaluse_list)
-				self.configdict["pkginternal"]["USE"] = pkginternaluse
-
-		feature_use = " ".join(feature_use)
-		if (feature_use != self.configdict["features"].get("USE", "") or
-			pkginternaluse is not pkginternaluse_before):
-			self.configdict["features"]["USE"] = feature_use
+		if restrict_test and "test" in self.features:
+			# Handle it like IUSE="-test", since features USE is
+			# independent of RESTRICT.
+			pkginternaluse_list.append("-test")
+			pkginternaluse = " ".join(pkginternaluse_list)
+			self.configdict["pkginternal"]["USE"] = pkginternaluse
 			# TODO: can we avoid that?
 			self.reset(keeping_pkg=1)
 			has_changed = True

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     01b8e9277b3f6970ea09135def26b70dbf072a16
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct  8 02:37:59 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct  8 04:29:34 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=01b8e927

_doebuild_path: Respect order defined in ROOTPATH (bug 667662)

Respect the order of paths defined in /etc/env.d/*, so that
packages like llvm can rely on ordering relative to paths
defined in /etc/env.d/50baselayout since baselayout-2.6.

See: https://gitweb.gentoo.org/proj/baselayout.git/commit/?id=277e5b9e55717873b87eb541a95f4f2ae0c60a4d
Bug: https://bugs.gentoo.org/667662
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 941a597e2..621fe7360 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -212,6 +212,7 @@ def _doebuild_path(settings, eapi=None):
 	eprefix = portage.const.EPREFIX
 	prerootpath = [x for x in settings.get("PREROOTPATH", "").split(":") if x]
 	rootpath = [x for x in settings.get("ROOTPATH", "").split(":") if x]
+	rootpath_set = frozenset(rootpath)
 	overrides = [x for x in settings.get(
 		"__PORTAGE_TEST_PATH_OVERRIDE", "").split(":") if x]
 
@@ -243,7 +244,10 @@ def _doebuild_path(settings, eapi=None):
 
 	for prefix in prefixes:
 		for x in ("usr/local/sbin", "usr/local/bin", "usr/sbin", "usr/bin", "sbin", "bin"):
-			path.append(os.path.join(prefix, x))
+			# Respect order defined in ROOTPATH
+			x_abs = os.path.join(prefix, x)
+			if x_abs not in rootpath_set:
+				path.append(x_abs)
 
 	path.extend(rootpath)
 	settings["PATH"] = ":".join(path)

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     1fc6cae0b2de181df5ba4c5b3f21b06604555387
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Wed Oct 10 07:52:10 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed Oct 10 07:57:38 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=1fc6cae0

doebuild: skip timestamp check for deleted distfiles (bug 668206)

Avoid behavior like this when a distfile has been removed:

>>> Existing ${T}/environment for 'chromium-70.0.3538.45' will be sourced.
>>> Run 'clean' to start with a fresh environment.
>>> Checking chromium-70.0.3538.45.tar.xz's mtime...
>>> Timestamp of chromium-70.0.3538.45.tar.xz has changed; recreating WORKDIR...

Bug: https://bugs.gentoo.org/668206
Reported-by: Michał Górny <mgorny <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 621fe7360..9706de422 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -934,10 +934,10 @@ def doebuild(myebuild, mydo, _unused=DeprecationWarning, settings=None, debug=0,
 							x_st = os.stat(os.path.join(
 								mysettings["DISTDIR"], x))
 						except OSError:
-							# file not fetched yet
+							# file deleted
 							x_st = None
 
-						if x_st is None or x_st.st_mtime > workdir_st.st_mtime:
+						if x_st is not None and x_st.st_mtime > workdir_st.st_mtime:
 							writemsg_stdout(_(">>> Timestamp of "
 								"%s has changed; recreating WORKDIR...\n") % x)
 							newstuff = True

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     4264ac4b07c4555eb5db2ab21288dba05a7f4b6d
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Nov  9 03:15:49 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Nov  9 03:21:55 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=4264ac4b

fetch: check DISTDIR write access as fetch user (bug 601252)

Test DISTDIR write access as the fetch user, for correct results
on NFS shares with root_squash enabled.

Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 0431e11ea..7ec7fe05e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -543,7 +543,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 
 	if can_fetch and \
 		not fetch_to_ro and \
-		not os.access(mysettings["DISTDIR"], os.W_OK):
+		_spawn_fetch(mysettings, ['test', '-w', mysettings["DISTDIR"]]) != os.EX_OK:
 		writemsg(_("!!! No write access to '%s'\n") % mysettings["DISTDIR"],
 			noiselevel=-1)
 		can_fetch = False

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     58eb01022e604b0a538b9d67bd64daa44da9f94a
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Nov 19 06:38:21 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Nov 19 06:42:06 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=58eb0102

Revert "fetch: check DISTDIR write access as fetch user (bug 601252)"

This reverts commit 4264ac4b07c4555eb5db2ab21288dba05a7f4b6d.
Dropping privileges for the access call is not useful by itself,
since lock and unlink calls do not drop privileges yet. We might
decide to drop privileges for the entire fetch function call.

Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 7ec7fe05e..0431e11ea 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -543,7 +543,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 
 	if can_fetch and \
 		not fetch_to_ro and \
-		_spawn_fetch(mysettings, ['test', '-w', mysettings["DISTDIR"]]) != os.EX_OK:
+		not os.access(mysettings["DISTDIR"], os.W_OK):
 		writemsg(_("!!! No write access to '%s'\n") % mysettings["DISTDIR"],
 			noiselevel=-1)
 		can_fetch = False

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     3ff6d9f92e16cc8b416a23486709adbd5bc13a06
Author:     Louis Sautier <sautier.louis <AT> gmail <DOT> com>
AuthorDate: Sun Nov 11 12:12:06 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Dec  7 00:40:05 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=3ff6d9f9

doebuild: add missing whitespace in warning message

Closes: https://github.com/gentoo/portage/pull/380
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 9917ac82c..5da9c9503 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -510,8 +510,8 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
 						mysettings["PATH"] = p + ":" + mysettings["PATH"]
 						break
 				else:
-					writemsg(("Warning: %s requested but no masquerade dir"
-						+ "can be found in /usr/lib*/%s/bin\n") % (m, m))
+					writemsg(("Warning: %s requested but no masquerade dir "
+						"can be found in /usr/lib*/%s/bin\n") % (m, m))
 					mysettings.features.remove(feature)
 
 		if 'MAKEOPTS' not in mysettings:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     22e8951521558a4488a44f4bdd276ced7c24b950
Author:     M. J. Everitt <m.j.everitt <AT> iee <DOT> org>
AuthorDate: Sat Dec 15 01:52:54 2018 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Thu Dec 20 04:24:31 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=22e89515

Provide compatibility support for old PORT_LOGDIR* variable names

Suggested-by: Zac Medico <zmedico <AT> gentoo.org>
Acked-by: Michael Everitt <m.j.everitt <AT> iee.org>
Bug: https://bugs.gentoo.org/668538
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index a68df5807..8de23e2e5 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -155,6 +155,9 @@ class config(object):
 	_constant_keys = frozenset(['PORTAGE_BIN_PATH', 'PORTAGE_GID',
 		'PORTAGE_PYM_PATH', 'PORTAGE_PYTHONPATH'])
 
+	_deprecated_keys = {'PORTAGE_LOGDIR': 'PORT_LOGDIR',
+		'PORTAGE_LOGDIR_CLEAN': 'PORT_LOGDIR_CLEAN'}
+
 	_setcpv_aux_keys = ('BDEPEND', 'DEFINED_PHASES', 'DEPEND', 'EAPI', 'HDEPEND',
 		'INHERITED', 'IUSE', 'REQUIRED_USE', 'KEYWORDS', 'LICENSE', 'PDEPEND',
 		'PROPERTIES', 'RDEPEND', 'SLOT',
@@ -2653,6 +2656,14 @@ class config(object):
 			except KeyError:
 				pass
 
+		deprecated_key = self._deprecated_keys.get(mykey)
+		if deprecated_key is not None:
+			value = self._getitem(deprecated_key)
+			#warnings.warn(_("Key %s has been renamed to %s. Please ",
+			#	"update your configuration") % (deprecated_key, mykey),
+			#	UserWarning)
+			return value
+
 		raise KeyError(mykey)
 
 	def get(self, k, x=None):

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     464a65b848eb1344cd1eff3545bed311c01d97ea
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Jan  1 20:55:41 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Jan  1 20:57:54 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=464a65b8

_check_temp_dir: fix message to refer to correct bug 378403

Bug: https://bugs.gentoo.org/378403
Fixes: 8a85160f4833 ("_check_temp_dir: check for 'portage' symlink")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 47c69967c..2dd458835 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1305,7 +1305,7 @@ def _check_temp_dir(settings):
 			msg = _("The 'portage' subdirectory of the directory "
 			"referenced by the PORTAGE_TMPDIR variable appears to be "
 			"a symlink. In order to avoid sandbox violations (see bug "
-			"#378379), you must adjust PORTAGE_TMPDIR instead of using "
+			"#378403), you must adjust PORTAGE_TMPDIR instead of using "
 			"the symlink located at '%s'. A suitable PORTAGE_TMPDIR "
 			"setting would be '%s'.") % \
 			(os.path.join(tmpdir, "portage"), checkdir)

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     e2eb3c8252c2ad154e84cc8803974103e9635eee
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Jan 21 22:11:43 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Jan 21 22:18:51 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=e2eb3c82

pid-sandbox: whitelist "depend" phase

Whitelist the "depend" phase for pid-sandbox, since this phase has very
little risk of leaving processes running, and pid-sandbox introduces
additional overhead that would affect egencache performance.

Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index f11923595..cf95dc406 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -111,7 +111,8 @@ _ipc_phases = frozenset([
 ])
 
 # phases which execute in the global PID namespace
-_global_pid_phases = frozenset(['preinst', 'postinst', 'prerm', 'postrm', 'config'])
+_global_pid_phases = frozenset([
+	'config', 'depend', 'preinst', 'prerm', 'postinst', 'postrm'])
 
 # phases in which networking access is allowed
 _networked_phases = frozenset([

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     c8e59b89b3d8f1fea20ec787c65d83becc0070ff
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Jun 10 18:28:56 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Jun 10 18:47:39 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=c8e59b89

config: don't swallow IOError for "packages" files (bug 687814)

Fixes: 55b2e1c0a51d ("Turn a traceback into a graceful error")
Bug: https://bugs.gentoo.org/687814
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index cc2413989..780013bca 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -51,6 +51,7 @@ from portage.util import ensure_dirs, getconfig, grabdict, \
 	grabdict_package, grabfile, grabfile_package, LazyItemsDict, \
 	normalize_path, shlex_split, stack_dictlist, stack_dicts, stack_lists, \
 	writemsg, writemsg_level, _eapi_cache
+from portage.util.install_mask import _raise_exc
 from portage.util.path import first_existing
 from portage.util._path import exists_raise_eaccess, isdir_raise_eaccess
 from portage.versions import catpkgsplit, catsplit, cpv_getkey, _pkg_str
@@ -596,10 +597,8 @@ class config(object):
 					verify_eapi=True, eapi=x.eapi, eapi_default=None,
 					allow_build_id=x.allow_build_id)
 					for x in profiles_complex]
-			except IOError as e:
-				if e.errno == IsADirectory.errno:
-					raise IsADirectory(os.path.join(self.profile_path,
-									 "packages"))
+			except EnvironmentError as e:
+				_raise_exc(e)
 
 			self.packages = tuple(stack_lists(packages_list, incremental=1))
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     f9f28734257129dd102d16e22ca5a6432243ddc3
Author:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Mon Jun 17 23:09:23 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Jun 18 17:11:06 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=f9f28734

ebuild: fix QA_SONAME_NO_SYMLINK

misc-functions.sh and ebuild.5 reference QA_SONAME_NO_SYMLINK.
Update doebuild.py to match this name.

Fixes: c9aebae4633a ("_post_src_install_soname_symlinks: fix bug 543818")
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index bea5535bd..67867d33e 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2263,7 +2263,7 @@ def _post_src_install_soname_symlinks(mysettings, out):
 			f.close()
 
 	metadata = {}
-	for k in ("QA_PREBUILT", "QA_NO_SYMLINK"):
+	for k in ("QA_PREBUILT", "QA_SONAME_NO_SYMLINK"):
 		try:
 			with io.open(_unicode_encode(os.path.join(
 				mysettings["PORTAGE_BUILDDIR"],
@@ -2284,14 +2284,14 @@ def _post_src_install_soname_symlinks(mysettings, out):
 			fnmatch.translate(x.lstrip(os.sep))
 			for x in portage.util.shlex_split(qa_prebuilt)))
 
-	qa_no_symlink = metadata.get("QA_NO_SYMLINK", "").split()
-	if qa_no_symlink:
-		if len(qa_no_symlink) > 1:
-			qa_no_symlink = "|".join("(%s)" % x for x in qa_no_symlink)
-			qa_no_symlink = "^(%s)$" % qa_no_symlink
+	qa_soname_no_symlink = metadata.get("QA_SONAME_NO_SYMLINK", "").split()
+	if qa_soname_no_symlink:
+		if len(qa_soname_no_symlink) > 1:
+			qa_soname_no_symlink = "|".join("(%s)" % x for x in qa_soname_no_symlink)
+			qa_soname_no_symlink = "^(%s)$" % qa_soname_no_symlink
 		else:
-			qa_no_symlink = "^%s$" % qa_no_symlink[0]
-		qa_no_symlink = re.compile(qa_no_symlink)
+			qa_soname_no_symlink = "^%s$" % qa_soname_no_symlink[0]
+		qa_soname_no_symlink = re.compile(qa_soname_no_symlink)
 
 	libpaths = set(portage.util.getlibpaths(
 		mysettings["ROOT"], env=mysettings))
@@ -2400,7 +2400,7 @@ def _post_src_install_soname_symlinks(mysettings, out):
 			continue
 		if not is_libdir(os.path.dirname(obj)):
 			continue
-		if qa_no_symlink and qa_no_symlink.match(obj.strip(os.sep)) is not None:
+		if qa_soname_no_symlink and qa_soname_no_symlink.match(obj.strip(os.sep)) is not None:
 			continue
 
 		obj_file_path = os.path.join(image_dir, obj.lstrip(os.sep))

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Ulrich Müller]

commit:     4fea794c94932a63942c0722183e1c2dffd6ecd2
Author:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
AuthorDate: Mon Jul 29 12:22:57 2019 +0000
Commit:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Tue Jul 30 07:07:10 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=4fea794c

doebuild.py: Override network-sandbox in unpack only for live ebuilds.

Suggested-by: Michał Górny <mgorny <AT> gentoo.org>
Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 67867d33e..6f980f87d 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -115,13 +115,6 @@ _ipc_phases = frozenset([
 _global_pid_phases = frozenset([
 	'config', 'depend', 'preinst', 'prerm', 'postinst', 'postrm'])
 
-# phases in which networking access is allowed
-_networked_phases = frozenset([
-	# for VCS fetching
-	"unpack",
-	# + for network-bound IPC
-] + list(_ipc_phases))
-
 _phase_func_map = {
 	"config": "pkg_config",
 	"setup": "pkg_setup",
@@ -156,7 +149,9 @@ def _doebuild_spawn(phase, settings, actionmap=None, **kwargs):
 		phase in _ipc_phases
 	kwargs['mountns'] = 'mount-sandbox' in settings.features
 	kwargs['networked'] = 'network-sandbox' not in settings.features or \
-		phase in _networked_phases or \
+		(phase == 'unpack' and \
+		'live' in settings.configdict['pkg'].get('PROPERTIES', '').split()) or \
+		phase in _ipc_phases or \
 		'network-sandbox' in settings['PORTAGE_RESTRICT'].split()
 	kwargs['pidns'] = ('pid-sandbox' in settings.features and
 		phase not in _global_pid_phases)

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     f28d32f298d4b089a2e36bebca2a55b6aeabe2a3
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun Aug 11 13:51:35 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed Aug 14 01:29:38 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=f28d32f2

Allow ESYSROOT and BROOT in the pkg_setup phase

This follows a recent change to PMS.

Signed-off-by: James Le Cuirot <chewi <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 83a15b370..e0dda54d4 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -2820,12 +2820,13 @@ class config(object):
 		if not eapi_exports_merge_type(eapi):
 			mydict.pop("MERGE_TYPE", None)
 
-		src_phase = _phase_func_map.get(phase, '').startswith('src_')
+		src_like_phase = (phase == 'setup' or
+				_phase_func_map.get(phase, '').startswith('src_'))
 
-		if not (src_phase and eapi_attrs.sysroot):
+		if not (src_like_phase and eapi_attrs.sysroot):
 			mydict.pop("ESYSROOT", None)
 
-		if not (src_phase and eapi_attrs.broot):
+		if not (src_like_phase and eapi_attrs.broot):
 			mydict.pop("BROOT", None)
 
 		# Prefix variables are supported beginning with EAPI 3, or when

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     26c8526293bd93906390d8fc830dc55052fc4d40
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 20 23:41:24 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Aug 20 23:42:53 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=26c85262

doebuild_environment: add newline to PORTAGE_COMPRESSION_COMMAND messages

Fixes: cff2c0149142 ("Support different compressors for binary packages")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 6f980f87d..584ff798b 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -543,7 +543,7 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
 			compression = _compressors[binpkg_compression]
 		except KeyError as e:
 			if binpkg_compression:
-				writemsg("Warning: Invalid or unsupported compression method: %s" % e.args[0])
+				writemsg("Warning: Invalid or unsupported compression method: %s\n" % e.args[0])
 			else:
 				# Empty BINPKG_COMPRESS disables compression.
 				mysettings['PORTAGE_COMPRESSION_COMMAND'] = 'cat'
@@ -551,11 +551,11 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
 			try:
 				compression_binary = shlex_split(varexpand(compression["compress"], mydict=settings))[0]
 			except IndexError as e:
-				writemsg("Warning: Invalid or unsupported compression method: %s" % e.args[0])
+				writemsg("Warning: Invalid or unsupported compression method: %s\n" % e.args[0])
 			else:
 				if find_binary(compression_binary) is None:
 					missing_package = compression["package"]
-					writemsg("Warning: File compression unsupported %s. Missing package: %s" % (binpkg_compression, missing_package))
+					writemsg("Warning: File compression unsupported %s. Missing package: %s\n" % (binpkg_compression, missing_package))
 				else:
 					cmd = [varexpand(x, mydict=settings) for x in shlex_split(compression["compress"])]
 					# Filter empty elements

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     7cb7196fe8aa668e76437a8a4500a7bcb9fd3c75
Author:     Arfrever Frehtes Taifersar Arahesis <Arfrever <AT> Apache <DOT> Org>
AuthorDate: Mon Sep  2 05:32:09 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Sep  2 20:11:26 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=7cb7196f

_doebuild_path: Optimize PATH in ebuild environment.

Canonicalize paths of directories in PATH.
Avoid duplicates when some directories (e.g. bin and sbin) are merged.

Bug: https://bugs.gentoo.org/693306
Signed-off-by: Arfrever Frehtes Taifersar Arahesis <Arfrever <AT> Apache.Org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 584ff798b..66e294d51 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -5,6 +5,7 @@ from __future__ import unicode_literals
 
 __all__ = ['doebuild', 'doebuild_environment', 'spawn', 'spawnebuild']
 
+import collections
 import grp
 import gzip
 import errno
@@ -254,7 +255,14 @@ def _doebuild_path(settings, eapi=None):
 				path.append(x_abs)
 
 	path.extend(rootpath)
-	settings["PATH"] = ":".join(path)
+
+	# Canonicalize paths and avoid duplicates when some directories
+	# (e.g. bin and sbin) are merged.
+	real_path = collections.OrderedDict()
+	for x in path:
+		real_path.setdefault(os.path.realpath(x), None)
+
+	settings["PATH"] = ":".join(real_path)
 
 def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
 	debug=False, use_cache=None, db=None):

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     c9c2bcc619c6ff60e452205f94ae26d4afd44d82
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Sep  3 15:34:08 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Sep  3 15:36:23 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=c9c2bcc6

Revert "_doebuild_path: Optimize PATH in ebuild environment."

This reverts commit 7cb7196fe8aa668e76437a8a4500a7bcb9fd3c75.
Since binary and installed packages have persistent PATH
settings in environment.bz2, using realpath on PATH components
can break binary and installed package phases if the symlink
layout is somehow modified. Therefore, the implications of
changes like this need to be considered very carefully.

Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 66e294d51..584ff798b 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -5,7 +5,6 @@ from __future__ import unicode_literals
 
 __all__ = ['doebuild', 'doebuild_environment', 'spawn', 'spawnebuild']
 
-import collections
 import grp
 import gzip
 import errno
@@ -255,14 +254,7 @@ def _doebuild_path(settings, eapi=None):
 				path.append(x_abs)
 
 	path.extend(rootpath)
-
-	# Canonicalize paths and avoid duplicates when some directories
-	# (e.g. bin and sbin) are merged.
-	real_path = collections.OrderedDict()
-	for x in path:
-		real_path.setdefault(os.path.realpath(x), None)
-
-	settings["PATH"] = ":".join(real_path)
+	settings["PATH"] = ":".join(path)
 
 def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
 	debug=False, use_cache=None, db=None):

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     0e8d17b56f5b86bd4b66d4720808e8b30c90a0ed
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun Oct 13 11:55:29 2019 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Sun Oct 13 19:49:49 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=0e8d17b5

fetch: Make FlatLayout.get_filenames() not return directories

Stop returning directories from FlatLayout.get_filenames().  This causes
emirrordist to wrongly presume directories created by new layout
to be distfiles, and causes some noisy errors.

Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 5d0bc7355..cd204b755 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -35,6 +35,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
 	'portage.util:atomic_ofstream',
 	'portage.util.configparser:SafeConfigParser,read_configs,' +
 		'ConfigParserError',
+	'portage.util.install_mask:_raise_exc',
 	'portage.util._urlopen:urlopen',
 )
 
@@ -269,7 +270,9 @@ class FlatLayout(object):
 		return filename
 
 	def get_filenames(self, distdir):
-		return iter(os.listdir(distdir))
+		for dirpath, dirnames, filenames in os.walk(distdir,
+				onerror=_raise_exc):
+			return iter(filenames)
 
 	@staticmethod
 	def verify_args(args):

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     7c0fd425fd33a75dc8978b5d9ef6d4bd7f8a31bc
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 14 20:10:41 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 14 20:11:40 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=7c0fd425

get_mirror_url: handle UnicodeDecodeError

Fixes: 6a539b7c5163 ("fetch: Support GLEP 75 mirror structure")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 1d5e07260..4e67dff97 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -419,7 +419,7 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
 				mirror_conf.read_from_file(data)
 			except ConfigParserError:
 				pass
-		except IOError:
+		except (IOError, UnicodeDecodeError):
 			pass
 
 		cache[mirror_url] = (time.time(), mirror_conf.serialize())

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     41ae830027648e65f867476a4039e7f51db64673
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 14 20:34:37 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 14 20:35:10 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=41ae8300

get_mirror_url: do not cache negative results

Bug: https://bugs.gentoo.org/697566
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 4e67dff97..107fba50e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -420,7 +420,8 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
 			except ConfigParserError:
 				pass
 		except (IOError, UnicodeDecodeError):
-			pass
+			# Do not cache negative results.
+			cache_path = None
 
 		cache[mirror_url] = (time.time(), mirror_conf.serialize())
 		if cache_path is not None:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     76fd5593d8ddf446f445ba3dd9da18a6e44dc5b3
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 14 20:37:25 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 14 20:38:29 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=76fd5593

get_mirror_url: do not cache after ConfigParserError

Bug: https://bugs.gentoo.org/697566
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 107fba50e..debab38a2 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -415,11 +415,8 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
 			finally:
 				f.close()
 
-			try:
-				mirror_conf.read_from_file(data)
-			except ConfigParserError:
-				pass
-		except (IOError, UnicodeDecodeError):
+			mirror_conf.read_from_file(data)
+		except (ConfigParserError, IOError, UnicodeDecodeError):
 			# Do not cache negative results.
 			cache_path = None
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     d4854fc4e4f3dc4578ae732a4aa9990933f47b7d
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Oct 19 23:48:56 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Oct 19 23:52:04 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=d4854fc4

fetch: add docstring

Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index debab38a2..76e4636c2 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -433,7 +433,31 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
 def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 	locks_in_subdir=".locks", use_locks=1, try_mirrors=1, digests=None,
 	allow_missing_digests=True):
-	"fetch files.  Will use digest file if available."
+	"""
+	Fetch files to DISTDIR and also verify digests if they are available.
+
+	@param myuris: Maps each file name to a tuple of available fetch URIs.
+	@type myuris: dict
+	@param mysettings: Portage config instance.
+	@type mysettings: portage.config
+	@param listonly: Only print URIs and do not actually fetch them.
+	@type listonly: bool
+	@param fetchonly: Do not block for files that are locked by a
+		concurrent fetcher process. This means that the function can
+		return successfully *before* all files have been successfully
+		fetched!
+	@type fetchonly: bool
+	@param use_locks: Enable locks. This parameter is ineffective if
+		FEATURES=distlocks is disabled in the portage config!
+	@type use_locks: bool
+	@param digests: Maps each file name to a dict of digest types and values.
+	@type digests: dict
+	@param allow_missing_digests: Enable fetch even if there are no digests
+		available for verification.
+	@type allow_missing_digests: bool
+	@rtype: int
+	@return: 1 if successful, 0 otherwise.
+	"""
 
 	if not myuris:
 		return 1

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     9115e1f6ba35cdcd85f2292dc293f0696caa8f12
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Fri Oct 18 07:23:02 2019 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Sun Oct 20 09:25:10 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=9115e1f6

fetch: Use distfile fetching method to get layout.conf

Rewrite the layout.conf getter to reuse the standard fetch() method
rather than using urlopen().  While at it, fix negative cache elision
to apply to memory cache as well (and not get written to disk if next
mirror was fine).

Most importantly, this ensures that we respect FETCHCOMMAND while
fetching layout.conf, and so layout.conf is fetched the same way normal
distfiles are.  With some uncommon configurations, the previous disjoint
logic might have resulted in one of the fetches failing while the other
succeeded.

This also adds some nice verbosity.  If mirror connection takes a while,
the user sees that rather than having Portage wait silently.

Bug: https://bugs.gentoo.org/697566
Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 33 ++++++++++++++++-----------------
 1 file changed, 16 insertions(+), 17 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 05de12740..cedf12b19 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -382,7 +382,7 @@ class MirrorLayoutConfig(object):
 		return ret
 
 
-def get_mirror_url(mirror_url, filename, cache_path=None):
+def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
 	"""
 	Get correct fetch URL for a given file, accounting for mirror
 	layout configuration.
@@ -408,23 +408,22 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
 	if ts >= time.time() - 86400:
 		mirror_conf.deserialize(data)
 	else:
+		tmpfile = '.layout.conf.%s' % urlparse(mirror_url).hostname
 		try:
-			f = urlopen(mirror_url + '/distfiles/layout.conf')
-			try:
-				data = io.StringIO(f.read().decode('utf8'))
-			finally:
-				f.close()
-
-			mirror_conf.read_from_file(data)
+			if fetch({tmpfile: (mirror_url + '/distfiles/layout.conf',)},
+					mysettings, force=1, try_mirrors=0):
+				tmpfile = os.path.join(mysettings['DISTDIR'], tmpfile)
+				mirror_conf.read_from_file(tmpfile)
+			else:
+				raise IOError()
 		except (ConfigParserError, IOError, UnicodeDecodeError):
-			# Do not cache negative results.
-			cache_path = None
-
-		cache[mirror_url] = (time.time(), mirror_conf.serialize())
-		if cache_path is not None:
-			f = atomic_ofstream(cache_path, 'w')
-			json.dump(cache, f)
-			f.close()
+			pass
+		else:
+			cache[mirror_url] = (time.time(), mirror_conf.serialize())
+			if cache_path is not None:
+				f = atomic_ofstream(cache_path, 'w')
+				json.dump(cache, f)
+				f.close()
 
 	return (mirror_url + "/distfiles/" +
 			mirror_conf.get_best_supported_layout().get_path(filename))
@@ -656,7 +655,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 				mirror_cache = None
 			for l in locations:
 				filedict[myfile].append(functools.partial(
-					get_mirror_url, l, myfile, mirror_cache))
+					get_mirror_url, l, myfile, mysettings, mirror_cache))
 		if myuri is None:
 			continue
 		if myuri[:9]=="mirror://":

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     db78fff2ffc15c5e245d87c2944c869dbe260a78
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 21 17:48:11 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 21 17:48:53 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=db78fff2

FilenameHashLayout.get_filenames: force UTF8 encoding for iglob argument

Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index ac25e4326..cf073436e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -307,7 +307,7 @@ class FilenameHashLayout(object):
 			c = c // 4
 			pattern += c * '[0-9a-f]' + '/'
 		pattern += '*'
-		for x in glob.iglob(os.path.join(distdir, pattern)):
+		for x in glob.iglob(portage._unicode_encode(os.path.join(distdir, pattern), errors='strict')):
 			try:
 				yield portage._unicode_decode(x, errors='strict').rsplit('/', 1)[1]
 			except UnicodeDecodeError:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     26fd7ffdd5b74af3aeedf0e6a87ac6b3d1243848
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Thu Oct 24 19:28:40 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Thu Oct 24 19:29:26 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=26fd7ffd

fetch: respect try_mirrors parameter for local mirrors

This is needed in order to prevent infinit recursion like this:

  File "portage/package/ebuild/fetch.py", line 1131, in fetch
    loc = loc()
  File "portage/package/ebuild/fetch.py", line 424, in get_mirror_url
    mysettings, force=1, try_mirrors=0):
  File "portage/package/ebuild/fetch.py", line 1131, in fetch
    loc = loc()

Bug: https://bugs.gentoo.org/698474
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index cf073436e..58718b568 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -588,7 +588,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 #			use_locks = 0
 
 	# local mirrors are always added
-	if "local" in custommirrors:
+	if try_mirrors and "local" in custommirrors:
 		mymirrors += custommirrors["local"]
 
 	if restrict_mirror:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     1ca5b822133171b131cef3dc15dc43583893ad6b
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Oct 29 00:56:47 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Oct 29 01:07:45 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=1ca5b822

fetch: remove symlink created by FETCHCOMMAND_RSYNC (bug 698046)

This avoids confusing "No such file or directory" errors as demonstrated
by the following test case:

$ ln -s /foo/bar /tmp/sudo-1.8.29rc1.tar.gz
$ wget http://distfiles.gentoo.org/distfiles/sudo-1.8.29rc1.tar.gz -O /tmp/sudo-1.8.29rc1.tar.gz
/tmp/sudo-1.8.29rc1.tar.gz: No such file or directory

Bug: https://bugs.gentoo.org/698046
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 58718b568..4612532c5 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1283,7 +1283,8 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 					# trust the return value from the fetcher.  Remove the
 					# empty file and try to download again.
 					try:
-						if os.stat(download_path).st_size == 0:
+						mystat = os.lstat(download_path)
+						if mystat.st_size == 0 or (stat.S_ISLNK(mystat.st_mode) and not os.path.exists(download_path)):
 							os.unlink(download_path)
 							fetched = 0
 							continue

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     2022595f042458e1561e0c3ee8f9f987eb5f27b0
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Dec  9 06:56:07 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Dec  9 07:00:30 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=2022595f

fetch: skip _checksum_failure_temp_file in force mode

Avoid files like this in DISTDIR:

$ md5sum .layout*
92dbdc707e5a40c7bfcbd1883173076a  .layout.conf.distfiles.gentoo.org
92dbdc707e5a40c7bfcbd1883173076a  .layout.conf.distfiles.gentoo.org._checksum_failure_.yrje0wk5

Reported-by: Arfrever Frehtes Taifersar Arahesis <Arfrever <AT> Apache.Org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 4612532c5..7ab054874 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -935,7 +935,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 							level=logging.ERROR, noiselevel=-1)
 						return 0
 
-					if distdir_writable:
+					if distdir_writable and not force:
 						# Since _check_distfile did not match above, the file
 						# is either corrupt or its identity has changed since
 						# the last time it was fetched, so rename it.

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     9b07545003fed649b1a0a8a9e5cb69d5e2fa4951
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 15 20:23:23 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Feb 15 20:25:34 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=9b075450

fetch: split out _ensure_distdir function (bug 601252)

Split out an _ensure_distdir function which will have to be
called earlier if the fetch function is called with dropped
privileges as discussed in bug 601252.

Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 102 +++++++++++++++++++-----------------
 1 file changed, 55 insertions(+), 47 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 7ab054874..e0260829b 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -152,6 +152,59 @@ def _userpriv_test_write_file(settings, file_path):
 	_userpriv_test_write_file_cache[file_path] = rval
 	return rval
 
+
+def _ensure_distdir(settings, distdir):
+	"""
+	Ensure that DISTDIR exists with appropriate permissions.
+
+	@param settings: portage config
+	@type settings: portage.package.ebuild.config.config
+	@param distdir: DISTDIR path
+	@type distdir: str
+	@raise PortageException: portage.exception wrapper exception
+	"""
+	global _userpriv_test_write_file_cache
+	dirmode  = 0o070
+	filemode =   0o60
+	modemask =    0o2
+	dir_gid = portage_gid
+	if "FAKED_MODE" in settings:
+		# When inside fakeroot, directories with portage's gid appear
+		# to have root's gid. Therefore, use root's gid instead of
+		# portage's gid to avoid spurrious permissions adjustments
+		# when inside fakeroot.
+		dir_gid = 0
+
+	userfetch = portage.data.secpass >= 2 and "userfetch" in settings.features
+	userpriv = portage.data.secpass >= 2 and "userpriv" in settings.features
+	write_test_file = os.path.join(distdir, ".__portage_test_write__")
+
+	try:
+		st = os.stat(distdir)
+	except OSError:
+		st = None
+
+	if st is not None and stat.S_ISDIR(st.st_mode):
+		if not (userfetch or userpriv):
+			return
+		if _userpriv_test_write_file(settings, write_test_file):
+			return
+
+	_userpriv_test_write_file_cache.pop(write_test_file, None)
+	if ensure_dirs(distdir, gid=dir_gid, mode=dirmode, mask=modemask):
+		if st is None:
+			# The directory has just been created
+			# and therefore it must be empty.
+			return
+		writemsg(_("Adjusting permissions recursively: '%s'\n") % distdir,
+			noiselevel=-1)
+		if not apply_recursive_permissions(distdir,
+			gid=dir_gid, dirmode=dirmode, dirmask=modemask,
+			filemode=filemode, filemask=modemask, onerror=_raise_exc):
+			raise OperationNotPermitted(
+				_("Failed to apply recursive permissions for the portage group."))
+
+
 def _checksum_failure_temp_file(settings, distdir, basename):
 	"""
 	First try to find a duplicate temp file with the same checksum and return
@@ -486,9 +539,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 
 	features = mysettings.features
 	restrict = mysettings.get("PORTAGE_RESTRICT","").split()
-
-	userfetch = secpass >= 2 and "userfetch" in features
-	userpriv = secpass >= 2 and "userpriv" in features
+	userfetch = portage.data.secpass >= 2 and "userfetch" in features
 
 	# 'nomirror' is bad/negative logic. You Restrict mirroring, not no-mirroring.
 	restrict_mirror = "mirror" in restrict or "nomirror" in restrict
@@ -728,51 +779,8 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 		can_fetch = False
 
 	if can_fetch and not fetch_to_ro:
-		global _userpriv_test_write_file_cache
-		dirmode  = 0o070
-		filemode =   0o60
-		modemask =    0o2
-		dir_gid = portage_gid
-		if "FAKED_MODE" in mysettings:
-			# When inside fakeroot, directories with portage's gid appear
-			# to have root's gid. Therefore, use root's gid instead of
-			# portage's gid to avoid spurrious permissions adjustments
-			# when inside fakeroot.
-			dir_gid = 0
-		distdir_dirs = [""]
 		try:
-			
-			for x in distdir_dirs:
-				mydir = os.path.join(mysettings["DISTDIR"], x)
-				write_test_file = os.path.join(
-					mydir, ".__portage_test_write__")
-
-				try:
-					st = os.stat(mydir)
-				except OSError:
-					st = None
-
-				if st is not None and stat.S_ISDIR(st.st_mode):
-					if not (userfetch or userpriv):
-						continue
-					if _userpriv_test_write_file(mysettings, write_test_file):
-						continue
-
-				_userpriv_test_write_file_cache.pop(write_test_file, None)
-				if ensure_dirs(mydir, gid=dir_gid, mode=dirmode, mask=modemask):
-					if st is None:
-						# The directory has just been created
-						# and therefore it must be empty.
-						continue
-					writemsg(_("Adjusting permissions recursively: '%s'\n") % mydir,
-						noiselevel=-1)
-					def onerror(e):
-						raise # bail out on the first error that occurs during recursion
-					if not apply_recursive_permissions(mydir,
-						gid=dir_gid, dirmode=dirmode, dirmask=modemask,
-						filemode=filemode, filemask=modemask, onerror=onerror):
-						raise OperationNotPermitted(
-							_("Failed to apply recursive permissions for the portage group."))
+			_ensure_distdir(mysettings, mysettings["DISTDIR"])
 		except PortageException as e:
 			if not os.path.isdir(mysettings["DISTDIR"]):
 				writemsg("!!! %s\n" % str(e), noiselevel=-1)

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     d8d02bd82f11921db5140941cef4077359b0211d
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 15 20:37:39 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Feb 15 20:42:24 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=d8d02bd8

fetch: remove secpass import (bug 601252)

Refer to the portage.secpass.data attribute directly rather than via
import value, since the value will change after fork and os.setuid
calls when the fetch function is called with dropped privileges as
discussed in bug 601252.

Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index e0260829b..06118b1a6 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1,4 +1,4 @@
-# Copyright 2010-2019 Gentoo Authors
+# Copyright 2010-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 from __future__ import print_function
@@ -46,7 +46,7 @@ from portage.checksum import (get_valid_checksum_keys, perform_md5, verify_all,
 	checksum_str)
 from portage.const import BASH_BINARY, CUSTOM_MIRRORS_FILE, \
 	GLOBAL_CONFIG_PATH
-from portage.data import portage_gid, portage_uid, secpass, userpriv_groups
+from portage.data import portage_gid, portage_uid, userpriv_groups
 from portage.exception import FileNotFound, OperationNotPermitted, \
 	PortageException, TryAgain
 from portage.localization import _
@@ -882,7 +882,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 
 					if not has_space_superuser:
 						has_space = False
-					elif secpass < 2:
+					elif portage.data.secpass < 2:
 						has_space = False
 					elif userfetch:
 						has_space = False

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     7adc9b05fc96382d583828f660c9c2e2c2323f22
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Mar  2 04:42:12 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Mar  2 04:43:57 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=7adc9b05

_check_build_log: convert zlib EOFError to eerror message (bug 711174)

Display a non-fatal eerror message when EOFError is encountered,
since this need not be fatal.

Bug: https://bugs.gentoo.org/711174
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 71e3a74ce..384a3e632 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1884,7 +1884,7 @@ def _check_build_log(mysettings, out=None):
 			if make_jobserver_re.match(line) is not None:
 				make_jobserver.append(line.rstrip("\n"))
 
-	except zlib.error as e:
+	except (EOFError, zlib.error) as e:
 		_eerror(["portage encountered a zlib error: '%s'" % (e,),
 			"while reading the log file: '%s'" % logfile])
 	finally:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     0bf72716cacc7f79abb61ad3ce33332c743809c9
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Mar  2 16:49:33 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Mar  2 16:51:15 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=0bf72716

doebuild: avoid emerge --pretend --fetchonly event loop recursion

Fixes: ec654122c0eb ("fetch: drop privileges early for NFS root_squash (bug 601252)")
Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 384a3e632..75fcb8a51 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1095,8 +1095,16 @@ def doebuild(myebuild, mydo, _unused=DeprecationWarning, settings=None, debug=0,
 					digests=dist_digests)
 
 			loop = asyncio._safe_loop()
-			if not loop.run_until_complete(loop.run_in_executor(ForkExecutor(loop=loop),
-				_fetch_subprocess, fetchme, mysettings, listonly, dist_digests)):
+			if loop.is_running():
+				# Called by EbuildFetchonly for emerge --pretend --fetchonly.
+				success = fetch(fetchme, mysettings, listonly=listonly,
+					fetchonly=fetchonly, allow_missing_digests=False,
+					digests=dist_digests)
+			else:
+				success = loop.run_until_complete(
+					loop.run_in_executor(ForkExecutor(loop=loop),
+					_fetch_subprocess, fetchme, mysettings, listonly, dist_digests))
+			if not success:
 				# Since listonly mode is called by emerge --pretend in an
 				# asynchronous context, spawn_nofetch would trigger event loop
 				# recursion here, therefore delegate execution of pkg_nofetch

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     ef8c21e59953aa5fdd153f19b7bf04356e2164fe
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Mar 15 00:01:25 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Mar 15 00:42:43 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=ef8c21e5

_prepare_workdir: apply PORTAGE_WORKDIR_MODE to PORTAGE_BUILDDIR (bug 692492)

Apply secure PORTAGE_WORKDIR_MODE permissions to PORTAGE_BUILDDIR,
since the child directory ${D} and its children may have vulnerable
permissions as reported in bug 692492.

Bug: https://bugs.gentoo.org/692492
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/prepare_build_dirs.py | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index c325819d1..8349d306f 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -1,4 +1,4 @@
-# Copyright 2010-2018 Gentoo Foundation
+# Copyright 2010-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 from __future__ import unicode_literals
@@ -84,7 +84,7 @@ def prepare_build_dirs(myroot=None, settings=None, cleanup=False):
 			except PortageException:
 				if not os.path.isdir(mydir):
 					raise
-		for dir_key in ("PORTAGE_BUILDDIR", "HOME", "PKG_LOGDIR", "T"):
+		for dir_key in ("HOME", "PKG_LOGDIR", "T"):
 			ensure_dirs(mysettings[dir_key], mode=0o755)
 			apply_secpass_permissions(mysettings[dir_key],
 				uid=portage_uid, gid=portage_gid)
@@ -272,11 +272,18 @@ def _prepare_workdir(mysettings):
 		writemsg(_("!!! Unable to parse PORTAGE_WORKDIR_MODE='%s', using %s.\n") % \
 		(mysettings["PORTAGE_WORKDIR_MODE"], oct(workdir_mode)))
 	mysettings["PORTAGE_WORKDIR_MODE"] = oct(workdir_mode).replace('o', '')
-	try:
-		apply_secpass_permissions(mysettings["WORKDIR"],
-		uid=portage_uid, gid=portage_gid, mode=workdir_mode)
-	except FileNotFound:
-		pass # ebuild.sh will create it
+
+	permissions = {'mode': workdir_mode}
+	if portage.data.secpass >= 2:
+		permissions['uid'] = portage_uid
+	if portage.data.secpass >= 1:
+		permissions['gid'] = portage_gid
+
+	# Apply PORTAGE_WORKDIR_MODE to PORTAGE_BUILDDIR, since the child
+	# directory ${D} and its children may have vulnerable permissions
+	# as reported in bug 692492.
+	ensure_dirs(mysettings["PORTAGE_BUILDDIR"], **permissions)
+	ensure_dirs(mysettings["WORKDIR"], **permissions)
 
 	if mysettings.get("PORTAGE_LOGDIR", "") == "":
 		while "PORTAGE_LOGDIR" in mysettings:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     2342e77eea8e00a572623f86d20cde2dc7098aa5
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Mar 22 01:13:22 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Mar 23 01:15:04 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=2342e77e

_post_phase_userpriv_perms: handle $HOME (bug 713100)

Ensure that the userpriv UID has appropriate permission for files
created in $HOME during privileged phases like pkg_setup, in the
same way as for $T. This prevents potential permission issues for
programs invoked during unprivileged phases, and it improves
alignment with PMS which specifies identical behavior for both
$HOME and $T.

Bug: https://bugs.gentoo.org/713100
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 75fcb8a51..2bff94cb1 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1765,9 +1765,10 @@ def _post_phase_userpriv_perms(mysettings):
 	if "userpriv" in mysettings.features and secpass >= 2:
 		""" Privileged phases may have left files that need to be made
 		writable to a less privileged user."""
-		apply_recursive_permissions(mysettings["T"],
-			uid=portage_uid, gid=portage_gid, dirmode=0o700, dirmask=0,
-			filemode=0o600, filemask=0)
+		for path in (mysettings["HOME"], mysettings["T"]):
+			apply_recursive_permissions(path,
+				uid=portage_uid, gid=portage_gid, dirmode=0o700, dirmask=0,
+				filemode=0o600, filemask=0)
 
 
 def _check_build_log(mysettings, out=None):

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     90b7d3b00d5a5eb9b732fb8b5bf0870eb89ab422
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Mar 22 21:42:42 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Mar 23 01:16:09 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=90b7d3b0

_hide_url_passwd: replace all occurrences (bug 713726)

Adjust the regular expression to avoid overly-greedy .+ groups,
so that is will properly replace all occurrences, as necessary
for the purposes of bug 713726 since PORTAGE_BINHOST may contain
multiple values.

Bug: https://bugs.gentoo.org/713726
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index f7984130f..28e7caf53 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -67,7 +67,7 @@ _userpriv_spawn_kwargs = (
 )
 
 def _hide_url_passwd(url):
-	return re.sub(r'//(.+):.+@(.+)', r'//\1:*password*@\2', url)
+	return re.sub(r'//([^:\s]+):[^@\s]+@', r'//\1:*password*@', url)
 
 
 def _want_userfetch(settings):

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     61f76236aa10c297f79c2fee123a79d26bfa9327
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Wed May  6 04:54:03 2020 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Wed May  6 06:37:12 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=61f76236

doebuild: Use ccache/distcc/icecream only in src_* phases

Enable ccache/distcc/icecream only when src_* phases are executed.
There is generally little value from them in pkg_*, as these phases
use CC only to do a few tests at most.  These compilations generally
are not run in parallel and the programs are tiny, so usually
the overhead exceeds the gain.

On the other hand, running them in pkg_* phases results in files being
created with root ownership that breaks them afterwards when using
userpriv.  As a result, user not only loses the benefit of, say, distcc
but also frequently hits error messages that confuse some configure
scripts and result in miscompiled packages.

Bug: https://bugs.gentoo.org/581880
Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Closes: https://github.com/gentoo/portage/pull/547
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 2bff94cb1..1343eca6d 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -486,7 +486,14 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
 		distcc = "distcc" in mysettings.features
 		icecream = "icecream" in mysettings.features
 
-		if ccache or distcc or icecream:
+		# run cc thingies only in src_* phases as otherwise they might
+		# create files with incorrect ownership and trip over
+		# (NB: install has a similar issue but 1) some badly written
+		# packages rebuild stuff there, so ccache is very helpful
+		# and 2) almost always some compilation will happen earlier,
+		# so permissions will be already set correctly)
+		if (ccache or distcc or icecream) and mydo in ('unpack',
+				'prepare', 'configure', 'test', 'install'):
 			libdir = None
 			default_abi = mysettings.get("DEFAULT_ABI")
 			if default_abi:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     878eddfea083eda5aea60510aa1c4e9145f26b99
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 19:22:17 2020 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Mon May 18 19:22:17 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=878eddfe

Revert "doebuild: Use ccache/distcc/icecream only in src_* phases"

This breaks ccache/etc. entirely as they are no longer present in PATH
in later phases.

Reverts: 61f76236aa10c297f79c2fee123a79d26bfa9327
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 1343eca6d..2bff94cb1 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -486,14 +486,7 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
 		distcc = "distcc" in mysettings.features
 		icecream = "icecream" in mysettings.features
 
-		# run cc thingies only in src_* phases as otherwise they might
-		# create files with incorrect ownership and trip over
-		# (NB: install has a similar issue but 1) some badly written
-		# packages rebuild stuff there, so ccache is very helpful
-		# and 2) almost always some compilation will happen earlier,
-		# so permissions will be already set correctly)
-		if (ccache or distcc or icecream) and mydo in ('unpack',
-				'prepare', 'configure', 'test', 'install'):
+		if ccache or distcc or icecream:
 			libdir = None
 			default_abi = mysettings.get("DEFAULT_ABI")
 			if default_abi:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Mike Gilbert]

commit:     4c18f523bb86a8be4c148f365dabee06fca2e4fa
Author:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Sun May 31 13:11:41 2020 +0000
Commit:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Sun May 31 20:32:46 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=4c18f523

Escape percent-signs in filename when fetching from mirrors

Bug: https://bugs.gentoo.org/719810
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 28e7caf53..47c3ad28f 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -26,6 +26,11 @@ try:
 except ImportError:
 	from urlparse import urlparse
 
+try:
+	from urllib.parse import quote as urlquote
+except ImportError:
+	from urllib import quote as urlquote
+
 import portage
 portage.proxy.lazyimport.lazyimport(globals(),
 	'portage.package.ebuild.config:check_config_instance,config',
@@ -351,7 +356,7 @@ _size_suffix_map = {
 
 class FlatLayout(object):
 	def get_path(self, filename):
-		return filename
+		return urlquote(filename)
 
 	def get_filenames(self, distdir):
 		for dirpath, dirnames, filenames in os.walk(distdir,
@@ -382,7 +387,7 @@ class FilenameHashLayout(object):
 			c = c // 4
 			ret += fnhash[:c] + '/'
 			fnhash = fnhash[c:]
-		return ret + filename
+		return ret + urlquote(filename)
 
 	def get_filenames(self, distdir):
 		pattern = ''

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Mike Gilbert]

commit:     54d572cbc3ef2b646283b826813a47e5268bf734
Author:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Sun May 31 21:15:29 2020 +0000
Commit:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Sun May 31 21:15:29 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=54d572cb

Revert "Escape percent-signs in filename when fetching from mirrors"

This probably breaks emirrordist; we will need separate methods for
layout.get_path() when it is used client-side.

This reverts commit 4c18f523bb86a8be4c148f365dabee06fca2e4fa.

Bug: https://bugs.gentoo.org/719810
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 47c3ad28f..28e7caf53 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -26,11 +26,6 @@ try:
 except ImportError:
 	from urlparse import urlparse
 
-try:
-	from urllib.parse import quote as urlquote
-except ImportError:
-	from urllib import quote as urlquote
-
 import portage
 portage.proxy.lazyimport.lazyimport(globals(),
 	'portage.package.ebuild.config:check_config_instance,config',
@@ -356,7 +351,7 @@ _size_suffix_map = {
 
 class FlatLayout(object):
 	def get_path(self, filename):
-		return urlquote(filename)
+		return filename
 
 	def get_filenames(self, distdir):
 		for dirpath, dirnames, filenames in os.walk(distdir,
@@ -387,7 +382,7 @@ class FilenameHashLayout(object):
 			c = c // 4
 			ret += fnhash[:c] + '/'
 			fnhash = fnhash[c:]
-		return ret + urlquote(filename)
+		return ret + filename
 
 	def get_filenames(self, distdir):
 		pattern = ''

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Mike Gilbert]

commit:     c238d5f7ed264179c263f5a2da983c4ee50b4f00
Author:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Sun May 31 21:24:25 2020 +0000
Commit:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Sun May 31 21:30:14 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=c238d5f7

Escape percent-signs in portage.package.ebuild.fetch.get_mirror_url()

This avoids double-escaping in emirrordist. We only want to escape the
path when fetching the file from the mirror, not when mirroring the
file.

Bug: https://bugs.gentoo.org/719810
Fixes: 4c18f523bb86a8be4c148f365dabee06fca2e4fa
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 28e7caf53..9682fea89 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -26,6 +26,11 @@ try:
 except ImportError:
 	from urlparse import urlparse
 
+try:
+	from urllib.parse import quote as urlquote
+except ImportError:
+	from urllib import quote as urlquote
+
 import portage
 portage.proxy.lazyimport.lazyimport(globals(),
 	'portage.package.ebuild.config:check_config_instance,config',
@@ -520,7 +525,7 @@ def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
 				f.close()
 
 	return (mirror_url + "/distfiles/" +
-			mirror_conf.get_best_supported_layout().get_path(filename))
+			urlquote(mirror_conf.get_best_supported_layout().get_path(filename)))
 
 
 def fetch(myuris, mysettings, listonly=0, fetchonly=0,

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     89b9dd50bc6b6cf0bb30c8caef2383734805caae
Author:     Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug  3 19:05:50 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug  3 19:21:57 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=89b9dd50

lib/portage/package/ebuild/getmaskingstatus.py: drop unused-import

Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/getmaskingstatus.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/lib/portage/package/ebuild/getmaskingstatus.py b/lib/portage/package/ebuild/getmaskingstatus.py
index cf2b7344b..c008a0d03 100644
--- a/lib/portage/package/ebuild/getmaskingstatus.py
+++ b/lib/portage/package/ebuild/getmaskingstatus.py
@@ -3,14 +3,12 @@
 
 __all__ = ['getmaskingstatus']
 
-
 import portage
 from portage import eapi_is_supported, _eapi_is_deprecated
 from portage.exception import InvalidDependString
 from portage.localization import _
 from portage.package.ebuild.config import config
-from portage.versions import catpkgsplit, _pkg_str
-
+from portage.versions import _pkg_str
 
 class _UnmaskHint:
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     ccb08d292503389635cf9183f79218ca02c12b01
Author:     Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug  3 19:05:45 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug  3 19:21:31 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=ccb08d29

lib/portage/package/ebuild/config.py: drop unused-import

* Fix unused-import
* Update copyright

Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 7c5f82e0c..74b998d0b 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -34,13 +34,11 @@ from portage.const import CACHE_PATH, \
 	PRIVATE_PATH, PROFILE_PATH, USER_CONFIG_PATH, \
 	USER_VIRTUALS_FILE
 from portage.dbapi import dbapi
-from portage.dbapi.porttree import portdbapi
 from portage.dep import Atom, isvalidatom, match_from_list, use_reduce, _repo_separator, _slot_separator
 from portage.eapi import (eapi_exports_AA, eapi_exports_merge_type,
 	eapi_supports_prefix, eapi_exports_replace_vars, _get_eapi_attrs)
 from portage.env.loaders import KeyValuePairFileLoader
-from portage.exception import InvalidDependString, IsADirectory, \
-		PortageException
+from portage.exception import InvalidDependString, PortageException
 from portage.localization import _
 from portage.output import colorize
 from portage.process import fakeroot_capable, sandbox_capable

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     5e7440acab6831b1e8c06c2f46794ec8b13b94ab
Author:     Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug  3 19:05:51 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug  3 19:21:54 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=5e7440ac

lib/portage/package/ebuild/_spawn_nofetch.py: drop unused-import

* Drop unused-import
* Update copyright

Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/_spawn_nofetch.py | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/lib/portage/package/ebuild/_spawn_nofetch.py b/lib/portage/package/ebuild/_spawn_nofetch.py
index 980d7f629..c7cf8d29b 100644
--- a/lib/portage/package/ebuild/_spawn_nofetch.py
+++ b/lib/portage/package/ebuild/_spawn_nofetch.py
@@ -1,9 +1,8 @@
-# Copyright 2010-2019 Gentoo Authors
+# Copyright 2010-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import tempfile
 
-import portage
 from portage import os
 from portage import shutil
 from portage.const import EBUILD_PHASES
@@ -13,12 +12,9 @@ from portage.package.ebuild.doebuild import doebuild_environment
 from portage.package.ebuild.prepare_build_dirs import prepare_build_dirs
 from portage.util.futures import asyncio
 from portage.util._async.SchedulerInterface import SchedulerInterface
-from portage.util._eventloop.EventLoop import EventLoop
-from portage.util._eventloop.global_event_loop import global_event_loop
 from _emerge.CompositeTask import CompositeTask
 from _emerge.EbuildPhase import EbuildPhase
 
-
 class SpawnNofetchWithoutBuilddir(CompositeTask):
 	"""
 	This spawns pkg_nofetch if appropriate, while avoiding the

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     06cd15d01685ab3f944ba0b9edade6ba67cdc2da
Author:     Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug  3 20:20:27 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug  3 21:25:53 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=06cd15d0

lib/portage/package/ebuild/prepare_build_dirs.py: drop unused-import

Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/prepare_build_dirs.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index b63cd89fa..080d5309f 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -18,7 +18,6 @@ from portage.output import colorize
 from portage.util import apply_recursive_permissions, \
 	apply_secpass_permissions, ensure_dirs, normalize_path, writemsg
 from portage.util.install_mask import _raise_exc
-from portage.const import EPREFIX
 
 def prepare_build_dirs(myroot=None, settings=None, cleanup=False):
 	"""

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     65a642eea83b50b8cf16a1cae5088db0a68383f2
Author:     Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug  3 20:20:28 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug  3 21:25:53 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=65a642ee

lib/portage/package/ebuild/getmaskingreason.py: drop unused-import

* Drop unused-import
* Update copyright

Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/getmaskingreason.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/getmaskingreason.py b/lib/portage/package/ebuild/getmaskingreason.py
index 22232de29..9cda20456 100644
--- a/lib/portage/package/ebuild/getmaskingreason.py
+++ b/lib/portage/package/ebuild/getmaskingreason.py
@@ -1,4 +1,4 @@
-# Copyright 2010-2014 Gentoo Foundation
+# Copyright 2010-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 __all__ = ['getmaskingreason']
@@ -10,7 +10,7 @@ from portage.dep import Atom, match_from_list
 from portage.exception import InvalidAtom
 from portage.localization import _
 from portage.repository.config import _gen_valid_repo
-from portage.util import grablines, normalize_path
+from portage.util import grablines
 from portage.versions import catpkgsplit, _pkg_str
 
 def getmaskingreason(mycpv, metadata=None, settings=None,

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     20b6a36fbf791bcc7d42bd429cf8116399a014a7
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Sep 12 21:23:28 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Sep 12 22:08:45 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=20b6a36f

get_mirror_url: urlquote only for ftp, http, and https (bug 741474)

It's necessary to use urlquote for correct behavior with ftp, http,
and https, since it's possible for file names to contain percent
encoded characters that need to be protected by an additional layer
of percent encoding. For other protocols such as sftp and rsync,
all characters are interpreted literally, so urlquote must not be
used.

Fixes: c238d5f7ed264179c263f5a2da983c4ee50b4f00
Bug: https://bugs.gentoo.org/719810
Bug: https://bugs.gentoo.org/741474
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 7c61fe463..ca031f31e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -513,8 +513,12 @@ def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
 				json.dump(cache, f)
 				f.close()
 
-	return (mirror_url + "/distfiles/" +
-			urlquote(mirror_conf.get_best_supported_layout().get_path(filename)))
+	# For some protocols, urlquote is required for correct behavior,
+	# and it must not be used for other protocols like rsync and sftp.
+	path = mirror_conf.get_best_supported_layout().get_path(filename)
+	if urlparse(mirror_url).scheme in ('ftp', 'http', 'https'):
+		path = urlquote(path)
+	return mirror_url + "/distfiles/" + path
 
 
 def fetch(myuris, mysettings, listonly=0, fetchonly=0,

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     e31c0c929673482101a066541d462d692502a6d1
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Nov  1 20:10:18 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Nov  2 01:30:32 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=e31c0c92

make.conf: expand special *ROOT variables (bug 752147)

Bug: https://bugs.gentoo.org/752147
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index a09fdbced..a188c700d 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -401,9 +401,14 @@ class config:
 			expand_map = env_d.copy()
 			self._expand_map = expand_map
 
-			# Allow make.globals to set default paths relative to ${EPREFIX}.
+			# Allow make.globals and make.conf to set paths relative to vars like ${EPREFIX}.
+			expand_map["BROOT"] = broot
 			expand_map["EPREFIX"] = eprefix
+			expand_map["EROOT"] = eroot
+			expand_map["ESYSROOT"] = esysroot
 			expand_map["PORTAGE_CONFIGROOT"] = config_root
+			expand_map["ROOT"] = target_root
+			expand_map["SYSROOT"] = sysroot
 
 			if portage._not_installed:
 				make_globals_path = os.path.join(PORTAGE_BASE_PATH, "cnf", "make.globals")

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     a362242c8a4e9e44c17c3d284dd2dbfc14e59635
Author:     Daniel M. Weeks <dan <AT> danweeks <DOT> net>
AuthorDate: Wed Nov 11 17:27:15 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Jan 10 03:19:10 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=a362242c

Fix varexpand, splitting in fetch

Performing variable expansion after splitting prevents expanding a
variable to an unquoted form:

Given VAR="a b c": "echo \${VAR}" produces ['echo', 'a b c']

It should be possible for a user to control quoting such that:

"echo \"\${VAR}\"" produces ['echo', 'a b c']
"echo \${VAR}" produces ['echo', 'a', 'b', 'c']

Reordering splitting and variable expansion fixes this and mirrors the
order used in _start_gpg_proc of ManifestTask.

Signed-off-by: Daniel M. Weeks <dan <AT> danweeks.net>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index ca031f31e..7be9d0239 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1296,8 +1296,9 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 						if v is not None:
 							variables[k] = v
 
-					myfetch = shlex_split(locfetch)
-					myfetch = [varexpand(x, mydict=variables) for x in myfetch]
+					myfetch = varexpand(locfetch, mydict=variables)
+					myfetch = shlex_split(myfetch)
+
 					myret = -1
 					try:
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     0aac11200780225cf14a07d6e4445234f1e0f72a
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 17 12:39:15 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Jan 17 13:02:29 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=0aac1120

doebuild: use NamedTemporaryFile for PORTAGE_BINPKG_TMPFILE

Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index f6cee4518..476689d5e 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1175,11 +1175,14 @@ def doebuild(myebuild, mydo, _unused=DeprecationWarning, settings=None, debug=0,
 				# the current user doesn't have write access to $PKGDIR.
 				if hasattr(portage, 'db'):
 					bintree = portage.db[mysettings['EROOT']]['bintree']
-					mysettings["PORTAGE_BINPKG_TMPFILE"] = \
-						bintree.getname(mysettings.mycpv) + \
-						".%s" % (portage.getpid(),)
-					bintree._ensure_dir(os.path.dirname(
-						mysettings["PORTAGE_BINPKG_TMPFILE"]))
+					binpkg_tmpfile_dir = os.path.join(bintree.pkgdir, mysettings["CATEGORY"])
+					bintree._ensure_dir(binpkg_tmpfile_dir)
+					with tempfile.NamedTemporaryFile(
+						prefix=mysettings["PF"],
+						suffix=".tbz2." + str(portage.getpid()),
+						dir=binpkg_tmpfile_dir,
+						delete=False) as binpkg_tmpfile:
+						mysettings["PORTAGE_BINPKG_TMPFILE"] = binpkg_tmpfile.name
 				else:
 					parent_dir = os.path.join(mysettings["PKGDIR"],
 						mysettings["CATEGORY"])

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     784d2c19f2d62982db16af9055bdab6b595a661b
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Feb 19 05:51:43 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Feb 22 05:26:53 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=784d2c19

make.defaults: prevent USE="${USE} ..." misbehavior

Discard parent profile USE from the expand_map variable before
it is used to evaluate a child profile. This prevents accidents
triggered by USE="${USE} ..." settings at the top of make.defaults
which caused parent profile USE to override parent profile
package.use settings.

Note that would be nice to guard the USE_EXPAND variables like
this too, but unfortunately USE_EXPAND is not known until after
make.defaults has been evaluated, so that will require some form
of make.defaults preprocessing.

Bug: https://bugs.gentoo.org/771549
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index e5ec681af..690efde9d 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -612,9 +612,20 @@ class config:
 
 			mygcfg = {}
 			if profiles_complex:
-				mygcfg_dlists = [getconfig(os.path.join(x.location, "make.defaults"),
-					tolerant=tolerant, expand=expand_map, recursive=x.portage1_directories)
-					for x in profiles_complex]
+				mygcfg_dlists = []
+				for x in profiles_complex:
+					# Prevent accidents triggered by USE="${USE} ..." settings
+					# at the top of make.defaults which caused parent profile
+					# USE to override parent profile package.use settings.
+					# It would be nice to guard USE_EXPAND variables like
+					# this too, but unfortunately USE_EXPAND is not known
+					# until after make.defaults has been evaluated, so that
+					# will require some form of make.defaults preprocessing.
+					expand_map.pop("USE", None)
+					mygcfg_dlists.append(
+						getconfig(os.path.join(x.location, "make.defaults"),
+						tolerant=tolerant, expand=expand_map,
+						recursive=x.portage1_directories))
 				self._make_defaults = mygcfg_dlists
 				mygcfg = stack_dicts(mygcfg_dlists,
 					incrementals=self.incrementals)

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     3e5ef19d5d6b26fde08da463d730ff90edf6eb29
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Wed Feb 24 15:08:30 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed Feb 24 15:12:40 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=3e5ef19d

fetch.py: fix UnboundLocalError: local variable 'myfile'

Reported-by: Ionen Wolkens <sudinave <AT> gmail.com>
Bug: https://bugs.gentoo.org/772386
Fixes: b9ef191c7498 ("MirrorLayoutConfig: content digest support (bug 756778)")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index f0ae864ad..d23bbfecb 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -898,11 +898,11 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 		for myuri in myuris:
 			if urlparse(myuri).scheme:
 				file_uri_tuples.append(
-					(DistfileName(myfile, digests=mydigests.get(myfile)), myuri)
+					(DistfileName(myfile, digests=mydigests.get(os.path.basename(myuri))), myuri)
 				)
 			else:
 				file_uri_tuples.append(
-					(DistfileName(myfile, digests=mydigests.get(myfile)), None)
+					(DistfileName(myfile, digests=mydigests.get(os.path.basename(myuri))), None)
 				)
 
 	filedict = OrderedDict()

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     48226b2df1408cf1944cb7c6128c73710c740dd3
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Thu Feb 25 09:26:49 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Thu Feb 25 09:31:01 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=48226b2d

fetch.py: fix UnboundLocalError: local variable 'myfile'

Reported-by: Ionen Wolkens <sudinave <AT> gmail.com>
Bug: https://bugs.gentoo.org/772386
Fixes: 3e5ef19d5d6b ("fetch.py: fix UnboundLocalError: local variable 'myfile'")
Fixes: b9ef191c7498 ("MirrorLayoutConfig: content digest support (bug 756778)")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index d23bbfecb..a683793f0 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -898,11 +898,11 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 		for myuri in myuris:
 			if urlparse(myuri).scheme:
 				file_uri_tuples.append(
-					(DistfileName(myfile, digests=mydigests.get(os.path.basename(myuri))), myuri)
+					(DistfileName(os.path.basename(myuri), digests=mydigests.get(os.path.basename(myuri))), myuri)
 				)
 			else:
 				file_uri_tuples.append(
-					(DistfileName(myfile, digests=mydigests.get(os.path.basename(myuri))), None)
+					(DistfileName(os.path.basename(myuri), digests=mydigests.get(os.path.basename(myuri))), None)
 				)
 
 	filedict = OrderedDict()

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     af0d91e9a251cb6ecf2568c51b21e6072e8c5241
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Wed May 12 21:37:56 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon May 24 04:56:02 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=af0d91e9

Refactor fetch() for selective restrictions

Refactor the fetch() function so that fetch/mirror restrictions only
affect the code per-URI rather than globally.  This will make it easier
to apply restrictions selectively in EAPI 8.

Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 54 ++++++++++++++++++-------------------
 1 file changed, 26 insertions(+), 28 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 73abec595..2f3111b65 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -813,8 +813,6 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 	custommirrors = grabdict(os.path.join(mysettings["PORTAGE_CONFIGROOT"],
 		CUSTOM_MIRRORS_FILE), recursive=1)
 
-	mymirrors=[]
-
 	if listonly or ("distlocks" not in features):
 		use_locks = 0
 
@@ -832,16 +830,22 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 				"FEATURES in /etc/portage/make.conf\n"), noiselevel=-1)
 #			use_locks = 0
 
-	# local mirrors are always added
-	if try_mirrors and "local" in custommirrors:
-		mymirrors += custommirrors["local"]
-
-	if restrict_mirror:
-		# We don't add any mirrors.
-		pass
-	else:
-		if try_mirrors:
-			mymirrors += [x.rstrip("/") for x in mysettings["GENTOO_MIRRORS"].split() if x]
+	local_mirrors = []
+	public_mirrors = []
+	fsmirrors = []
+	if try_mirrors:
+		for x in custommirrors.get("local", []):
+			if x.startswith("/"):
+				fsmirrors.append(x)
+			else:
+				local_mirrors.append(x)
+		for x in mysettings["GENTOO_MIRRORS"].split():
+			if not x:
+				continue
+			if x.startswith('/'):
+				fsmirrors.append(x.rstrip("/"))
+			else:
+				public_mirrors.append(x.rstrip("/"))
 
 	hash_filter = _hash_filter(mysettings.get("PORTAGE_CHECKSUM_FILTER", ""))
 	if hash_filter.transparent:
@@ -864,23 +868,8 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 		shlex_split(mysettings.get("PORTAGE_RO_DISTDIRS", "")) \
 		if os.path.isdir(x)]
 
-	fsmirrors = []
-	for x in range(len(mymirrors)-1,-1,-1):
-		if mymirrors[x] and mymirrors[x][0]=='/':
-			fsmirrors += [mymirrors[x]]
-			del mymirrors[x]
-
 	restrict_fetch = "fetch" in restrict
 	force_mirror = "force-mirror" in features and not restrict_mirror
-	custom_local_mirrors = custommirrors.get("local", [])
-	if restrict_fetch:
-		# With fetch restriction, a normal uri may only be fetched from
-		# custom local mirrors (if available).  A mirror:// uri may also
-		# be fetched from specific mirrors (effectively overriding fetch
-		# restriction, but only for specific mirrors).
-		locations = custom_local_mirrors
-	else:
-		locations = mymirrors
 
 	file_uri_tuples = []
 	# Check for 'items' attribute since OrderedDict is not a dict.
@@ -916,7 +905,16 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 						".mirror-cache.json")
 			else:
 				mirror_cache = None
-			for l in locations:
+
+			# With fetch restriction, a normal uri may only be fetched from
+			# custom local mirrors (if available).  A mirror:// uri may also
+			# be fetched from specific mirrors (effectively overriding fetch
+			# restriction, but only for specific mirrors).
+			location_lists = [local_mirrors]
+			if not restrict_fetch and not restrict_mirror:
+				location_lists.append(public_mirrors)
+
+			for l in itertools.chain(*location_lists):
 				filedict[myfile].append(functools.partial(
 					get_mirror_url, l, myfile, mysettings, mirror_cache))
 		if myuri is None:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     91e5c15943a8ad01f5133f21730d8800c415a00a
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sat May 29 10:16:59 2021 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Mon May 31 19:53:34 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=91e5c159

Emit QA notices for setuptools warnings

Emit QA notices for setuptools warnings such as:

     * QA Notice: setuptools warnings detected:
     *
     *      Usage of dash-separated 'build-requires' will not be supported in future versions. Please use the underscore name 'build_requires' instead
     *      Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
     *      Usage of dash-separated 'upload-dir' will not be supported in future versions. Please use the underscore name 'upload_dir' instead

Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Closes: https://github.com/gentoo/portage/pull/722
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 86c1d40b4..b1557edd7 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1863,6 +1863,10 @@ def _check_build_log(mysettings, out=None):
 		re.compile(r'g?make\[\d+\]: warning: jobserver unavailable:')
 	make_jobserver = []
 
+	# we deduplicate these since they is repeated for every setup.py call
+	setuptools_warn = set()
+	setuptools_warn_re = re.compile(r'.*\/setuptools\/.*: UserWarning: (.*)')
+
 	def _eerror(lines):
 		for line in lines:
 			eerror(line, phase="install", key=mysettings.mycpv, out=out)
@@ -1892,6 +1896,10 @@ def _check_build_log(mysettings, out=None):
 			if make_jobserver_re.match(line) is not None:
 				make_jobserver.append(line.rstrip("\n"))
 
+			m = setuptools_warn_re.match(line)
+			if m is not None:
+				setuptools_warn.add(m.group(1))
+
 	except (EOFError, zlib.error) as e:
 		_eerror(["portage encountered a zlib error: '%s'" % (e,),
 			"while reading the log file: '%s'" % logfile])
@@ -1945,6 +1953,12 @@ def _check_build_log(mysettings, out=None):
 		msg.extend("\t" + line for line in make_jobserver)
 		_eqawarn(msg)
 
+	if setuptools_warn:
+		msg = [_("QA Notice: setuptools warnings detected:")]
+		msg.append("")
+		msg.extend("\t" + line for line in sorted(setuptools_warn))
+		_eqawarn(msg)
+
 	f.close()
 	if f_real is not None:
 		f_real.close()

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     de877dba8af0d57dcf3767a891daf3fd6908d7ff
Author:     Daniel M. Weeks <dan <AT> danweeks <DOT> net>
AuthorDate: Mon May 31 15:46:55 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Jun  5 17:59:56 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=de877dba

Support GLEP 75 layout in GENTOO_MIRRORS

Signed-off-by: Daniel M. Weeks <dan <AT> danweeks.net>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index c2bfca61c..c2f7dffc0 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1253,7 +1253,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 
 				if fsmirrors and not os.path.exists(myfile_path) and has_space:
 					for mydir in fsmirrors:
-						mirror_file = os.path.join(mydir, myfile)
+						mirror_file = get_mirror_url(mydir, myfile, mysettings)
 						try:
 							shutil.copyfile(mirror_file, download_path)
 							writemsg(_("Local mirror has file: %s\n") % myfile)

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     e284848704cb5e189eec35a93a54a8e91aa42d64
Author:     Daniel M. Weeks <dan <AT> danweeks <DOT> net>
AuthorDate: Mon May 31 15:21:45 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Jun  5 17:59:55 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=e2848487

Support GLEP 75 layout in PORTAGE_RO_DISTDIRS

Signed-off-by: Daniel M. Weeks <dan <AT> danweeks.net>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index f687a01a2..c2bfca61c 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -663,8 +663,15 @@ def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
 	else:
 		tmpfile = '.layout.conf.%s' % urlparse(mirror_url).hostname
 		try:
-			if fetch({tmpfile: (mirror_url + '/distfiles/layout.conf',)},
-					mysettings, force=1, try_mirrors=0):
+			if mirror_url[:1] == "/":
+				tmpfile = os.path.join(mirror_url, "layout.conf")
+				mirror_conf.read_from_file(tmpfile)
+			elif fetch(
+				{tmpfile: (mirror_url + "/distfiles/layout.conf",)},
+				mysettings,
+				force=1,
+				try_mirrors=0,
+			):
 				tmpfile = os.path.join(mysettings['DISTDIR'], tmpfile)
 				mirror_conf.read_from_file(tmpfile)
 			else:
@@ -683,8 +690,10 @@ def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
 	path = mirror_conf.get_best_supported_layout(filename=filename).get_path(filename)
 	if urlparse(mirror_url).scheme in ('ftp', 'http', 'https'):
 		path = urlquote(path)
-	return mirror_url + "/distfiles/" + path
-
+	if mirror_url[:1] == "/":
+		return os.path.join(mirror_url, path)
+	else:
+		return mirror_url + "/distfiles/" + path
 
 def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 	locks_in_subdir=".locks", use_locks=1, try_mirrors=1, digests=None,
@@ -1212,7 +1221,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 				if distdir_writable and ro_distdirs:
 					readonly_file = None
 					for x in ro_distdirs:
-						filename = os.path.join(x, myfile)
+						filename = get_mirror_url(x, myfile, mysettings)
 						match, mystat = _check_distfile(
 							filename, pruned_digests, eout, hash_filter=hash_filter)
 						if match:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     2ce11f06e48290efb2d4b6743c8edf01c176b0fc
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun Jun 20 17:53:02 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Jun 20 18:54:21 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=2ce11f06

fetch: Fix AttributeError when no URL specified

Closes: https://github.com/gentoo/portage/pull/732
Bug: https://bugs.gentoo.org/796812
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index c2f7dffc0..100450792 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -907,8 +907,8 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
 	primaryuri_dict = {}
 	thirdpartymirror_uris = {}
 	for myfile, myuri in file_uri_tuples:
-		override_mirror = myuri.startswith("mirror+")
-		override_fetch = override_mirror or myuri.startswith("fetch+")
+		override_mirror = (myuri or "").startswith("mirror+")
+		override_fetch = override_mirror or (myuri or "").startswith("fetch+")
 		if override_fetch:
 			myuri = myuri.partition("+")[2]
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     95dbf3516c5c1f16212e4d87d42198896e8dade4
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Tue Jul  6 07:47:30 2021 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Thu Aug  5 08:46:53 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=95dbf351

Ignore version normalization warnings from setuptools

The version normalization warnings are not really meaningful and are
quite common when using date-based versioning.  Do not report them.

Closes: https://github.com/gentoo/portage/pull/735
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 0cbc2d01b..366cbb9ca 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1871,6 +1871,10 @@ def _check_build_log(mysettings, out=None):
 	# we deduplicate these since they is repeated for every setup.py call
 	setuptools_warn = set()
 	setuptools_warn_re = re.compile(r'.*\/setuptools\/.*: UserWarning: (.*)')
+	# skip useless version normalization warnings
+	setuptools_warn_ignore_re = [
+		re.compile(r'Normalizing .*')
+	]
 
 	def _eerror(lines):
 		for line in lines:
@@ -1903,7 +1907,12 @@ def _check_build_log(mysettings, out=None):
 
 			m = setuptools_warn_re.match(line)
 			if m is not None:
-				setuptools_warn.add(m.group(1))
+				warn_text = m.group(1)
+				for ign in setuptools_warn_ignore_re:
+					if ign.match(warn_text):
+						break
+				else:
+					setuptools_warn.add(warn_text)
 
 	except (EOFError, zlib.error) as e:
 		_eerror(["portage encountered a zlib error: '%s'" % (e,),

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     505b7d7fc997e71ebe6de17bda24a210f0e65042
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Wed Sep  8 09:32:35 2021 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Wed Sep  8 09:32:35 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=505b7d7f

Extend setuptools warning regexp

Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 366cbb9ca..3da239be2 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1870,7 +1870,7 @@ def _check_build_log(mysettings, out=None):
 
 	# we deduplicate these since they is repeated for every setup.py call
 	setuptools_warn = set()
-	setuptools_warn_re = re.compile(r'.*\/setuptools\/.*: UserWarning: (.*)')
+	setuptools_warn_re = re.compile(r'.*\/setuptools\/.*: .*Warning: (.*)')
 	# skip useless version normalization warnings
 	setuptools_warn_ignore_re = [
 		re.compile(r'Normalizing .*')

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     4c2d9475fe14a8947f4f552a1be0b101bb1eef96
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Mon Sep 27 20:50:53 2021 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Mon Sep 27 20:50:53 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=4c2d9475

prepare_build_dirs: Fix copytree() on Python 3.7

Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/prepare_build_dirs.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index 2e2ef73f4..f37c9647b 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -483,9 +483,9 @@ def _prepare_fake_filesdir(settings):
     portage.util.ensure_dirs(filesdir, mode=0o755)
 
     # Copy files from real directory to ebuild directory (without metadata).
-    if os.path.isdir(real_filesdir):
+    if os.path.isdir(real_filesdir) and not os.path.isdir(filesdir):
         shutil.copytree(
-            real_filesdir, filesdir, copy_function=copyfile, dirs_exist_ok=True
+            real_filesdir, filesdir, copy_function=copyfile
         )
 
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     6b2f67eddf66eb685cccfad3bec23147f318f420
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Sep 28 00:17:05 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Sep 28 07:19:47 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=6b2f67ed

prepare_build_dirs: apply permissions to filesdir

Bug: https://bugs.gentoo.org/815196
Reviewed-by: Michał Górny <mgorny <AT> gentoo.org>
Reviewed-by: Sam James <sam <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/prepare_build_dirs.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index f37c9647b..7407122ca 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -484,8 +484,15 @@ def _prepare_fake_filesdir(settings):
 
     # Copy files from real directory to ebuild directory (without metadata).
     if os.path.isdir(real_filesdir) and not os.path.isdir(filesdir):
-        shutil.copytree(
-            real_filesdir, filesdir, copy_function=copyfile
+        shutil.copytree(real_filesdir, filesdir, copy_function=copyfile)
+        apply_recursive_permissions(
+            filesdir,
+            uid=portage_uid,
+            gid=portage_gid,
+            dirmode=0o750,
+            dirmask=0,
+            filemode=0o640,
+            filemask=0,
         )
 
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     cc7d98cec7b410d875d44a5130b22960746aca06
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Tue Sep 28 11:19:20 2021 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Tue Sep 28 11:19:20 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=cc7d98ce

Revert "prepare_build_dirs: Fix copytree() on Python 3.7"

This is (obviously) the wrong fix, resulting in empty FILESDIR.

Reverts: 4c2d9475fe14a8947f4f552a1be0b101bb1eef96
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/prepare_build_dirs.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index 7407122ca..0ae93a1d5 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -483,7 +483,7 @@ def _prepare_fake_filesdir(settings):
     portage.util.ensure_dirs(filesdir, mode=0o755)
 
     # Copy files from real directory to ebuild directory (without metadata).
-    if os.path.isdir(real_filesdir) and not os.path.isdir(filesdir):
+    if os.path.isdir(real_filesdir):
         shutil.copytree(real_filesdir, filesdir, copy_function=copyfile)
         apply_recursive_permissions(
             filesdir,

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     773ba1701f94bdd46086d294efcf97985b67841d
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Tue Sep 28 11:23:16 2021 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Tue Sep 28 11:23:16 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=773ba170

Attempt to fix creating FILESDIR properly

Since we perform a mode fixup on FILESDIR anyway, just let copytree()
create it.  This should finally fix all the problems: have the directory
created without errors and work with Python < 3.8.

Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/prepare_build_dirs.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index 0ae93a1d5..41de1a3a9 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -480,7 +480,6 @@ def _ensure_log_subdirs(logdir, subdir):
 def _prepare_fake_filesdir(settings):
     real_filesdir = settings["O"] + "/files"
     filesdir = settings["FILESDIR"]
-    portage.util.ensure_dirs(filesdir, mode=0o755)
 
     # Copy files from real directory to ebuild directory (without metadata).
     if os.path.isdir(real_filesdir):

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     f371f1804c5d2d263ca78aa79c3feed1bc5831ae
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Oct  3 17:48:48 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Oct  3 19:30:33 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=f371f180

prepare_build_dirs: handle copytree FileExistsError

Bug: https://bugs.gentoo.org/815871
Reviewed-by: Michał Górny <mgorny <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/prepare_build_dirs.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index 41de1a3a9..410c7e4ae 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -477,13 +477,21 @@ def _ensure_log_subdirs(logdir, subdir):
         ensure_dirs(current, uid=uid, gid=gid, mode=grp_mode, mask=0)
 
 
+def _copytree(src, dst, **kwargs):
+    try:
+        shutil.copytree(src, dst, **kwargs)
+    except FileExistsError:
+        shutil.rmtree(dst)
+        shutil.copytree(src, dst, **kwargs)
+
+
 def _prepare_fake_filesdir(settings):
     real_filesdir = settings["O"] + "/files"
     filesdir = settings["FILESDIR"]
 
     # Copy files from real directory to ebuild directory (without metadata).
     if os.path.isdir(real_filesdir):
-        shutil.copytree(real_filesdir, filesdir, copy_function=copyfile)
+        _copytree(real_filesdir, filesdir, copy_function=copyfile)
         apply_recursive_permissions(
             filesdir,
             uid=portage_uid,

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     98a8a4af2c77477c327b22621b8a0aed6d15f574
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Thu Oct 28 04:40:09 2021 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Thu Oct 28 04:52:22 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=98a8a4af

doebuild.py: check for inconsistent PROVIDES/image post-src_install

This is part of a series of fixes for the linked bug (failure
to preserve libraries in some situations).

At the point of installation (even if not merging), we need
to detect inconsistent metadata: PROVIDES should be populated
if we're installing any dynamic libraries. This suggests that
e.g. scanelf malfunctioned or some corruption occurred.

Bug: https://bugs.gentoo.org/811462
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 9650a8444..96e56b277 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -3,6 +3,7 @@
 
 __all__ = ["doebuild", "doebuild_environment", "spawn", "spawnebuild"]
 
+import glob
 import grp
 import gzip
 import errno
@@ -118,6 +119,7 @@ from portage.util.futures import asyncio
 from portage.util.futures.executor.fork import ForkExecutor
 from portage.util.path import first_existing
 from portage.util.socks5 import get_socks5_proxy
+from portage.util._dyn_libs.dyn_libs import check_dyn_libs_inconsistent
 from portage.versions import _pkgsplit
 from _emerge.BinpkgEnvExtractor import BinpkgEnvExtractor
 from _emerge.EbuildBuildDir import EbuildBuildDir
@@ -126,7 +128,6 @@ from _emerge.EbuildSpawnProcess import EbuildSpawnProcess
 from _emerge.Package import Package
 from _emerge.RootConfig import RootConfig
 
-
 _unsandboxed_phases = frozenset(
     [
         "clean",
@@ -3091,6 +3092,16 @@ def _post_src_install_soname_symlinks(mysettings, out):
             errors="strict",
         ) as f:
             f.write(soname_deps.provides)
+    else:
+        if check_dyn_libs_inconsistent(image_dir, soname_deps.provides):
+            self._writemsg_level(
+                colorize(
+                    "BAD",
+                    "!!! Error! Installing dynamic libraries (.so) with blank PROVIDES!",
+                ),
+                noiselevel=-1,
+                level=logging.ERROR,
+            )
 
     if unrecognized_elf_files:
         qa_msg = ["QA Notice: Unrecognized ELF file(s):"]

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     e7fa8671f71cae3ea18be510f708b3c63877becd
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Thu Oct 28 04:59:39 2021 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Thu Oct 28 05:00:11 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=e7fa8671

lib/portage/package/ebuild/doebuild.py: drop unused import

Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 96e56b277..7706bf8cb 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -3,7 +3,6 @@
 
 __all__ = ["doebuild", "doebuild_environment", "spawn", "spawnebuild"]
 
-import glob
 import grp
 import gzip
 import errno

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     776ede149514db41c9c08276594b43b2682ba1a4
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Nov  5 04:08:19 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Nov  5 04:22:48 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=776ede14

fetch: enable resume for digestgen case

Enable resume for the digestgen case (no digests available), when
the temporary file exceeds PORTAGE_FETCH_RESUME_MIN_SIZE. This
fixes a case which caused the ebuild digest command to skip the
download and fail with a message like this:

!!! File b'/var/cache/distfiles/foo.tar.xz' doesn't exist, can't update Manifest

Bug: https://bugs.gentoo.org/821571
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 8c64362c2..2d3625800 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1485,10 +1485,11 @@ def fetch(
                             except EnvironmentError:
                                 pass
                     elif not orig_digests:
-                        # We don't have a digest, but the file exists.  We must
-                        # assume that it is fully downloaded.
+                        # We don't have a digest, and the temporary file exists.
                         if not force:
-                            continue
+                            # Try to resume this download when full
+                            # download has not been explicitly forced.
+                            fetched = 1
                     else:
                         if (
                             mydigests[myfile].get("size") is not None

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Michał Górny]

commit:     c309328c4e1f6254251d31149ee47b4266d4d70f
Author:     Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Mon Nov 15 08:31:30 2021 +0000
Commit:     Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Mon Nov 15 08:31:30 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=c309328c

package.ebuild.doebuild: ignore setuptools install deprecation

Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 7706bf8cb..34720c7c6 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2359,7 +2359,10 @@ def _check_build_log(mysettings, out=None):
     setuptools_warn = set()
     setuptools_warn_re = re.compile(r".*\/setuptools\/.*: .*Warning: (.*)")
     # skip useless version normalization warnings
-    setuptools_warn_ignore_re = [re.compile(r"Normalizing .*")]
+    setuptools_warn_ignore_re = [
+        re.compile(r"Normalizing .*"),
+        re.compile(r"setup.py install is deprecated"),
+    ]
 
     def _eerror(lines):
         for line in lines:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     c36f878509108c970907dc391d6562297881b400
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Sat Dec 11 02:47:50 2021 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sat Dec 11 02:54:00 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=c36f8785

lib/portage/package/ebuild/doebuild.py: use 'eerror' for blank PROVIDES message

This means that it's much harder to miss the error.

Bug: https://bugs.gentoo.org/811462
Reported-by: Brian Dolbec <dolsen <AT> gentoo.org>
Thanks-to: Arfrever Frehtes Taifersar Arahesis <Arfrever <AT> Apache.Org>
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 0ff64e275..ac627f555 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -3096,13 +3096,11 @@ def _post_src_install_soname_symlinks(mysettings, out):
             f.write(soname_deps.provides)
     else:
         if check_dyn_libs_inconsistent(image_dir, soname_deps.provides):
-            portage.util._writemsg_level(
-                colorize(
-                    "BAD",
-                    "!!! Error! Installing dynamic libraries (.so) with blank PROVIDES!",
-                ),
-                noiselevel=-1,
-                level=logging.ERROR,
+            eerror(
+                "Error! Installing dynamic libraries (.so) with blank PROVIDES!",
+                phase="install",
+                key=mysettings.mycpv,
+                out=out,
             )
 
     if unrecognized_elf_files:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     80a90acd3da4f24df4d6f66b070c785616c1ee34
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri Dec 10 22:50:05 2021 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sat Dec 11 02:53:59 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=80a90acd

lib/portage/package/ebuild/doebuild.py: fix undefined self

Fixes: 98a8a4af2c77477c327b22621b8a0aed6d15f574
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 34720c7c6..0ff64e275 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -3096,7 +3096,7 @@ def _post_src_install_soname_symlinks(mysettings, out):
             f.write(soname_deps.provides)
     else:
         if check_dyn_libs_inconsistent(image_dir, soname_deps.provides):
-            self._writemsg_level(
+            portage.util._writemsg_level(
                 colorize(
                     "BAD",
                     "!!! Error! Installing dynamic libraries (.so) with blank PROVIDES!",

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     e0c1a5badef219cc2247c0fd6548776e753c073b
Author:     Rin Cat (鈴猫) <syu.os <AT> protonmail <DOT> com>
AuthorDate: Sun Jun 12 18:18:07 2022 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sun Jun 12 18:25:33 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=e0c1a5ba

Fix variable typo

Signed-off-by: Sheng Yu <syu.os <AT> protonmail.com>
Closes: https://github.com/gentoo/portage/pull/839
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/config.py   | 2 +-
 lib/portage/package/ebuild/doebuild.py | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index f35fcdf65..49a906559 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1510,7 +1510,7 @@ class config:
             if binpkg_format not in SUPPORTED_GENTOO_BINPKG_FORMATS:
                 writemsg(
                     "!!! BINPKG_FORMAT contains invalid or "
-                    "unsupported format: %s" % binpkg_fotmat,
+                    "unsupported format: %s" % binpkg_format,
                     noiselevel=-1,
                 )
 

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 2251cdb8d..8ee9f73c6 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -647,7 +647,7 @@ def doebuild_environment(
         if binpkg_format not in portage.const.SUPPORTED_GENTOO_BINPKG_FORMATS:
             writemsg(
                 "!!! BINPKG_FORMAT contains invalid or "
-                "unsupported format: %s" % binpkg_fotmat,
+                "unsupported format: %s" % binpkg_format,
                 noiselevel=-1,
             )
             binpkg_format = "xpak"

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     7cc778fbf3aa7919036f2bb224b118d96c14f31e
Author:     Arfrever Frehtes Taifersar Arahesis <arfrever <AT> apache <DOT> org>
AuthorDate: Mon Jul 11 02:54:26 2022 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Tue Jul 12 23:30:05 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=7cc778fb

env-update: handle BROOT vs ROOT distinction for e.g. PATH

When Portage builds packages with a non-trivial ROOT setting, say,
ROOT=/tmp/stage1root, the build dependencies (compilers, tools, etc.)
should be taken from the actual build environment BROOT, say BROOT=/ ,
and that must also be reflected in the environment.

An example situation that leads to problems otherwise is the following:
* We build a clang-based stage1 with CC=clang
* The clang binary exists only in a custom directory pulled into PATH
  via env.d.
* ROOT=/tmp/stage1root is initially empty, meaning portage tries to
  build with settings from an empty env.d, and suddently cannot find
  the clang that perfectly exists in BROOT=/ anymore.
* The build fails.

The patch takes a more versatile approach and separates environment
variables into three categories:
* constructed from ROOT
* constructed from BROOT
* combined from ROOT and BROOT (checks EROOT, then BROOT. If present in both,
only EROOT is used.)

Thanks-to: Andreas K. Hüttel <dilfridge <AT> gentoo.org>
Thanks-to: James Le Cuirot <chewi <AT> gentoo.org>
Signed-off-by: Sam James <sam <AT> gentoo.org>
Closes: https://github.com/gentoo/portage/pull/851
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 77 +++++++++++++++++++++++++++++-------
 1 file changed, 62 insertions(+), 15 deletions(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 49a906559..2530c9d31 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -485,14 +485,7 @@ class config:
             # interaction with the calling environment that might
             # lead to unexpected results.
 
-            env_d = (
-                getconfig(
-                    os.path.join(eroot, "etc", "profile.env"),
-                    tolerant=tolerant,
-                    expand=False,
-                )
-                or {}
-            )
+            env_d = self._get_env_d(broot=broot, eroot=eroot, tolerant=tolerant)
             expand_map = env_d.copy()
             self._expand_map = expand_map
 
@@ -1209,6 +1202,61 @@ class config:
         if mycpv:
             self.setcpv(mycpv)
 
+    def _get_env_d(self, broot, eroot, tolerant):
+        broot_only_variables = (
+            "PATH",
+            "PREROOTPATH",
+            "ROOTPATH",
+        )
+        eroot_only_variables = (
+            "CONFIG_PROTECT",
+            "CONFIG_PROTECT_MASK",
+            "INFODIR",
+            "INFOPATH",
+            "MANPATH",
+            "PKG_CONFIG_.*",
+        )
+
+        broot_only_variables_re = re.compile(r"^(%s)$" % "|".join(broot_only_variables))
+        eroot_only_variables_re = re.compile(r"^(%s)$" % "|".join(eroot_only_variables))
+
+        broot_env_d_path = os.path.join(broot or "/", "etc", "profile.env")
+        eroot_env_d_path = os.path.join(eroot or "/", "etc", "profile.env")
+
+        if (
+            os.path.exists(broot_env_d_path)
+            and os.path.exists(eroot_env_d_path)
+            and os.path.samefile(broot_env_d_path, eroot_env_d_path)
+        ):
+            broot_env_d = (
+                getconfig(broot_env_d_path, tolerant=tolerant, expand=False) or {}
+            )
+            eroot_env_d = broot_env_d
+        else:
+            broot_env_d = (
+                getconfig(broot_env_d_path, tolerant=tolerant, expand=False) or {}
+            )
+            eroot_env_d = (
+                getconfig(eroot_env_d_path, tolerant=tolerant, expand=False) or {}
+            )
+
+        env_d = {}
+
+        for k in broot_env_d.keys() | eroot_env_d.keys():
+            if broot_only_variables_re.match(k):
+                if k in broot_env_d:
+                    env_d[k] = broot_env_d[k]
+            elif eroot_only_variables_re.match(k):
+                if k in eroot_env_d:
+                    env_d[k] = eroot_env_d[k]
+            else:
+                if k in eroot_env_d:
+                    env_d[k] = eroot_env_d[k]
+                elif k in broot_env_d:
+                    env_d[k] = broot_env_d[k]
+
+        return env_d
+
     def _init_iuse(self):
         self._iuse_effective = self._calc_iuse_effective()
         self._iuse_implicit_match = _iuse_implicit_match_cache(self)
@@ -2640,14 +2688,13 @@ class config:
 
     def reload(self):
         """Reload things like /etc/profile.env that can change during runtime."""
-        env_d_filename = os.path.join(self["EROOT"], "etc", "profile.env")
         self.configdict["env.d"].clear()
-        env_d = getconfig(env_d_filename, tolerant=self._tolerant, expand=False)
-        if env_d:
-            # env_d will be None if profile.env doesn't exist.
-            for k in self._env_d_blacklist:
-                env_d.pop(k, None)
-            self.configdict["env.d"].update(env_d)
+        env_d = self._get_env_d(
+            broot=self["BROOT"], eroot=self["EROOT"], tolerant=self._tolerant
+        )
+        for k in self._env_d_blacklist:
+            env_d.pop(k, None)
+        self.configdict["env.d"].update(env_d)
 
     def regenerate(self, useonly=0, use_cache=None):
         """

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Mike Gilbert]

commit:     69f2221413852630b2c231774e1f811f350fa3e6
Author:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Sat Dec 31 22:01:08 2022 +0000
Commit:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Mon Jan  2 20:35:27 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=69f22214

_doebuild_path: simplify logic used to set PATH

Remove logic to incoporate PREROOTPATH and ROOTPATH. I'm not sure where
PREROOTPATH was ever used, and ROOTPATH has been deprecated in
baselayout for a while.

Remove logic to add hard-coded directories like
${EPREFIX}/{,usr{,/local}/{sbin,bin}. Adding these paths is unnecessary
if env.d or the calling environment have a valid PATH setting.

Add logic to ignore PATH from the calling environment if PATH is set in
env.d. This ensures that packages can update PATH by installing files in
/etc/env.d and this will work without having to restart Portage with an
updated environment.

Bug: https://bugs.gentoo.org/607696
Bug: https://bugs.gentoo.org/693308
Bug: https://bugs.gentoo.org/888543
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 44 +++++++++++-----------------------
 1 file changed, 14 insertions(+), 30 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 05336e2aa..d29451efa 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -284,20 +284,8 @@ def _doebuild_path(settings, eapi=None):
     if portage_bin_path[0] != portage.const.PORTAGE_BIN_PATH:
         # Add a fallback path for restarting failed builds (bug 547086)
         portage_bin_path.append(portage.const.PORTAGE_BIN_PATH)
-    prerootpath = [x for x in settings.get("PREROOTPATH", "").split(":") if x]
-    rootpath = [x for x in settings.get("ROOTPATH", "").split(":") if x]
-    rootpath_set = frozenset(rootpath)
-    overrides = [
-        x for x in settings.get("__PORTAGE_TEST_PATH_OVERRIDE", "").split(":") if x
-    ]
-
-    prefixes = []
-    # settings["EPREFIX"] should take priority over portage.const.EPREFIX
-    if portage.const.EPREFIX != settings["EPREFIX"] and settings["ROOT"] == os.sep:
-        prefixes.append(settings["EPREFIX"])
-    prefixes.append(portage.const.EPREFIX)
 
-    path = overrides
+    path = [x for x in settings.get("__PORTAGE_TEST_PATH_OVERRIDE", "").split(":") if x]
 
     if "xattr" in settings.features:
         for x in portage_bin_path:
@@ -317,24 +305,20 @@ def _doebuild_path(settings, eapi=None):
 
     for x in portage_bin_path:
         path.append(os.path.join(x, "ebuild-helpers"))
-    path.extend(prerootpath)
-
-    for prefix in prefixes:
-        prefix = prefix if prefix else "/"
-        for x in (
-            "usr/local/sbin",
-            "usr/local/bin",
-            "usr/sbin",
-            "usr/bin",
-            "sbin",
-            "bin",
-        ):
-            # Respect order defined in ROOTPATH
-            x_abs = os.path.join(prefix, x)
-            if x_abs not in rootpath_set:
-                path.append(x_abs)
 
-    path.extend(rootpath)
+    # If PATH is set in env.d, ignore PATH from the calling environment.
+    # This allows packages to update our PATH as they get installed.
+    if "PATH" in settings.configdict["env.d"]:
+        settings.configdict["env"].pop("PATH", None)
+
+    if "PATH" in settings:
+        pathset = set(path)
+        for p in settings["PATH"].split(":"):
+            # Avoid duplicate entries.
+            if p not in pathset:
+                path.append(p)
+                pathset.add(p)
+
     settings["PATH"] = ":".join(path)
 
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Ulrich Müller]

commit:     fe3d9c11ba3460e8c5677ebfe45399e3396ea8d9
Author:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
AuthorDate: Fri Jan 27 08:37:21 2023 +0000
Commit:     Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Fri Jan 27 08:37:21 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=fe3d9c11

package/ebuild/config.py: Move a comment to make black happy

Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index f66bf6485..9ed59eb74 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -3347,8 +3347,8 @@ class config:
         if not (src_like_phase and eapi_attrs.broot):
             mydict.pop("BROOT", None)
 
-        # Prefix variables are supported beginning with EAPI 3.
         if phase == "depend" or (
+            # Prefix variables are supported beginning with EAPI 3.
             eapi is not None
             and not eapi_supports_prefix(eapi)
         ):

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     010d688e6bd9bb19460b920896549e67a8cb5004
Author:     Oskari Pirhonen <xxc3ncoredxx <AT> gmail <DOT> com>
AuthorDate: Sat Nov 19 07:02:21 2022 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Feb 17 00:52:11 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=010d688e

ebuild/doebuild.py: use -Oline if MAKEOPTS unset

If `MAKEOPTS` is unset in `make.conf`, add
`GNUMAKEFLAGS="--output-sync=line"` to the environment to ensure proper
synchronization of output when doing parallel builds. `GNUMAKEFLAGS` is
used to avoid portability issues from passing the non-standard option
directly to `make`.

Signed-off-by: Oskari Pirhonen <xxc3ncoredxx <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index b72d61c7a..2226812e3 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -592,6 +592,8 @@ def doebuild_environment(
             nproc = get_cpu_count()
             if nproc:
                 mysettings["MAKEOPTS"] = "-j%d" % (nproc)
+            if "GNUMAKEFLAGS" not in mysettings:
+                mysettings["GNUMAKEFLAGS"] = "--output-sync=line"
 
         if not eapi_exports_KV(eapi):
             # Discard KV for EAPIs that don't support it. Cached KV is restored

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Mike Gilbert]

commit:     f5ae3c4b198f0d5543afbdd36d923422cd451272
Author:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Tue Jun  6 18:15:36 2023 +0000
Commit:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Wed Jun 14 19:21:55 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=f5ae3c4b

fetch: do not rely on os.stat() for DISTDIR write test

If DISTDIR is an automount, os.stat() will not trigger it.
Just try to create a file instead.

Bug: https://bugs.gentoo.org/485100
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>

 lib/portage/package/ebuild/fetch.py | 17 ++++++-----------
 1 file changed, 6 insertions(+), 11 deletions(-)

diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 0d30cb88e..0ecad162e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -240,20 +240,15 @@ def _ensure_distdir(settings, distdir):
     userpriv = portage.data.secpass >= 2 and "userpriv" in settings.features
     write_test_file = os.path.join(distdir, ".__portage_test_write__")
 
-    try:
-        st = os.stat(distdir)
-    except OSError:
-        st = None
-
-    if st is not None and stat.S_ISDIR(st.st_mode):
-        if not (userfetch or userpriv):
-            return
-        if _userpriv_test_write_file(settings, write_test_file):
-            return
+    if _userpriv_test_write_file(settings, write_test_file):
+        return
 
     _userpriv_test_write_file_cache.pop(write_test_file, None)
+
+    already_exists = os.path.isdir(distdir)
+
     if ensure_dirs(distdir, gid=dir_gid, mode=dirmode, mask=modemask):
-        if st is None:
+        if not already_exists:
             # The directory has just been created
             # and therefore it must be empty.
             return

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Mike Gilbert]

commit:     37b108a7a4630583921484c0b5f513a7e384d851
Author:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Tue Jun  6 18:03:50 2023 +0000
Commit:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Wed Jun 14 19:21:01 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=37b108a7

doebuild: do not rely on os.access() for PORTAGE_TMPDIR write check

Calling os.access() on ${PORTAGE_TMPDIR}/portage will not trigger any
automount that the user may have configured there.

Instead, just try to create a file and catch PermissionError.

Bug: https://bugs.gentoo.org/890812
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index d30c6b8f0..8b65a7862 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1608,7 +1608,22 @@ def _check_temp_dir(settings):
     # for those people.
     checkdir = first_existing(os.path.join(settings["PORTAGE_TMPDIR"], "portage"))
 
-    if not os.access(checkdir, os.W_OK):
+    try:
+        with tempfile.NamedTemporaryFile(prefix="exectest-", dir=checkdir) as fd:
+            os.chmod(fd.name, 0o755)
+            if not os.access(fd.name, os.X_OK):
+                writemsg(
+                    _(
+                        "Can not execute files in %s\n"
+                        "Likely cause is that you've mounted it with one of the\n"
+                        "following mount options: 'noexec', 'user', 'users'\n\n"
+                        "Please make sure that portage can execute files in this directory.\n"
+                    )
+                    % checkdir,
+                    noiselevel=-1,
+                )
+                return 1
+    except PermissionError:
         writemsg(
             _(
                 "%s is not writable.\n"
@@ -1619,21 +1634,6 @@ def _check_temp_dir(settings):
         )
         return 1
 
-    with tempfile.NamedTemporaryFile(prefix="exectest-", dir=checkdir) as fd:
-        os.chmod(fd.name, 0o755)
-        if not os.access(fd.name, os.X_OK):
-            writemsg(
-                _(
-                    "Can not execute files in %s\n"
-                    "Likely cause is that you've mounted it with one of the\n"
-                    "following mount options: 'noexec', 'user', 'users'\n\n"
-                    "Please make sure that portage can execute files in this directory.\n"
-                )
-                % checkdir,
-                noiselevel=-1,
-            )
-            return 1
-
     return os.EX_OK
 
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     87d0a7447f1464157f18a8ba0991719777003325
Author:     Oskari Pirhonen <xxc3ncoredxx <AT> gmail <DOT> com>
AuthorDate: Sun Aug 13 03:11:15 2023 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sat Aug 19 15:05:19 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=87d0a744

doebuild: gate -Oline behind MAKEFLAGS check

Add a check to see if MAKEFLAGS is set (alongside the existing MAKEOPTS
and GNUMAKEFLAGS checks) before enabling output synchronization.

This is only tangentially related to the bug below via being a (likely
rare) case of MAKEFLAGS being set in the wild.

Bug: https://bugs.gentoo.org/909009
Signed-off-by: Oskari Pirhonen <xxc3ncoredxx <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 8b65a7862f..4d4e674c4b 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -589,7 +589,7 @@ def doebuild_environment(
             nproc = get_cpu_count()
             if nproc:
                 mysettings["MAKEOPTS"] = "-j%d" % (nproc)
-            if "GNUMAKEFLAGS" not in mysettings:
+            if "GNUMAKEFLAGS" not in mysettings and "MAKEFLAGS" not in mysettings:
                 mysettings["GNUMAKEFLAGS"] = "--output-sync=line"
 
         if not eapi_exports_KV(eapi):

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     c16ff6913b3d0a4d4ea0f6ee811ece541865bd8f
Author:     Raul E Rangel <rrangel <AT> chromium <DOT> org>
AuthorDate: Wed Oct 11 18:42:34 2023 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Oct 13 10:19:01 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=c16ff691

config: Sort USE_EXPAND variables

Without the sort, we end up extending the USE_EXPAND variables with
a non deterministic ordering. This makes binary package creation non
hermetic.

i.e.,
```
-declare -x PYTHON_TARGETS="python3_8 python3_7 python3_6 python3_10 python3_9"
+declare -x PYTHON_TARGETS="python3_8 python3_9 python3_10 python3_7 python3_6"
```

Assuming `PYTHON_TARGETS=python3_8` in make.conf, after this change we
get:
```
declare -x PYTHON_TARGETS="python3_8 python3_10 python3_6 python3_7 python3_9"
```

Ideally we would completely sort the USE_EXPAND variables, but the
LINGUAS variable appears to need a very specific ordering.

Bug: https://bugs.gentoo.org/914441
Signed-off-by: Raul E Rangel <rrangel <AT> chromium.org>
Closes: https://github.com/gentoo/portage/pull/1098
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 3ee7fd65ed..049c5fa169 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1746,7 +1746,7 @@ class config:
             # Preserve the order of var_split because it can matter for things
             # like LINGUAS.
             var_split = [x for x in var_split if x in expand_flags]
-            var_split.extend(expand_flags.difference(var_split))
+            var_split.extend(sorted(expand_flags.difference(var_split)))
             has_wildcard = "*" in expand_flags
             if has_wildcard:
                 var_split = [x for x in var_split if x != "*"]

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     8c2d81eb205bf05445eca616818681e047639eb5
Author:     Raul E Rangel <rrangel <AT> chromium <DOT> org>
AuthorDate: Tue Sep 19 20:43:21 2023 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Oct 13 10:19:00 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=8c2d81eb

config: Don't directly modify FEATURES

Iterating on self.features returns items in a non-deterministic order,
it also leaves self.features out of sync. If we instead use the `remove`
method, the values get sorted and synchronized correctly.

Bug: https://bugs.gentoo.org/914441
Signed-off-by: Raul E Rangel <rrangel <AT> chromium.org>
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index e07d27e8fc..3ee7fd65ed 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -2204,7 +2204,7 @@ class config:
                 # "test" is in IUSE and USE=test is masked, so execution
                 # of src_test() probably is not reliable. Therefore,
                 # temporarily disable FEATURES=test just for this package.
-                self["FEATURES"] = " ".join(x for x in self.features if x != "test")
+                self.features.remove("test")
 
         # Allow _* flags from USE_EXPAND wildcards to pass through here.
         use.difference_update(

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     b150419d28bd7afb98404a829c639584d34efc03
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri Oct 13 10:32:26 2023 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Oct 13 10:33:13 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=b150419d

Revert "config: Don't directly modify FEATURES"

This reverts commit 8c2d81eb205bf05445eca616818681e047639eb5.

I get a bunch of lost USE=test with this:
```
$ emerge -p -uvDU @world
[...]
[ebuild   R    ] dev-libs/tree-sitter-bash-0.20.4-r1::gentoo  USE="-test*" 0 KiB
[ebuild   R    ] dev-libs/libffi-3.4.4-r1:0/8::gentoo  USE="-debug (-exec-static-trampoline) -pax-kernel -static-libs -test*" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] dev-util/pkgconf-2.0.3:0/4::gentoo  USE="-test*" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] dev-libs/libpipeline-1.5.7::gentoo  USE="-test*" 0 KiB
[ebuild   R    ] dev-util/fq-0.8.0::gentoo  USE="-test*" 0 KiB
[...]
```

Bug: https://bugs.gentoo.org/914441
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 049c5fa169..f51886c33e 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -2204,7 +2204,7 @@ class config:
                 # "test" is in IUSE and USE=test is masked, so execution
                 # of src_test() probably is not reliable. Therefore,
                 # temporarily disable FEATURES=test just for this package.
-                self.features.remove("test")
+                self["FEATURES"] = " ".join(x for x in self.features if x != "test")
 
         # Allow _* flags from USE_EXPAND wildcards to pass through here.
         use.difference_update(

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     ea1c12ac91faeb25b30d364afa1764506b7a1535
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 16 04:48:10 2023 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 16 05:14:08 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=ea1c12ac

_post_src_install_uid_fix: Pass unicode to write_atomic for utf8_mode

Bug: https://bugs.gentoo.org/915837
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 5780c2b0b3..b967fc061f 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2777,7 +2777,9 @@ def _post_src_install_uid_fix(mysettings, out):
                         # a normal write might fail due to file permission
                         # settings on some operating systems such as HP-UX
                         write_atomic(
-                            _unicode_encode(
+                            fpath
+                            if portage.utf8_mode
+                            else _unicode_encode(
                                 fpath, encoding=_encodings["merge"], errors="strict"
                             ),
                             new_contents,

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     e9d6782ca18b2f30d97b4859419bf7da4da01bab
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Oct 22 21:24:47 2023 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Oct 22 22:45:02 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=e9d6782c

doebuild: Make _fetch_subprocess compatible with spawn

Before it failed like this with the multiprocessing spawn
start method:

AttributeError: Can't pickle local object 'doebuild.<locals>._fetch_subprocess'

Bug: https://bugs.gentoo.org/916139
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 32 +++++++++++++++++---------------
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index b967fc061f..ba166173a8 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1305,21 +1305,6 @@ def doebuild(
             if mf is not None:
                 dist_digests = mf.getTypeDigests("DIST")
 
-            def _fetch_subprocess(fetchme, mysettings, listonly, dist_digests):
-                # For userfetch, drop privileges for the entire fetch call, in
-                # order to handle DISTDIR on NFS with root_squash for bug 601252.
-                if _want_userfetch(mysettings):
-                    _drop_privs_userfetch(mysettings)
-
-                return fetch(
-                    fetchme,
-                    mysettings,
-                    listonly=listonly,
-                    fetchonly=fetchonly,
-                    allow_missing_digests=False,
-                    digests=dist_digests,
-                )
-
             loop = asyncio._safe_loop()
             if loop.is_running():
                 # Called by EbuildFetchonly for emerge --pretend --fetchonly.
@@ -1340,6 +1325,7 @@ def doebuild(
                         mysettings,
                         listonly,
                         dist_digests,
+                        fetchonly,
                     )
                 )
             if not success:
@@ -1589,6 +1575,22 @@ def doebuild(
             portage._doebuild_manifest_exempt_depend -= 1
 
 
+def _fetch_subprocess(fetchme, mysettings, listonly, dist_digests, fetchonly):
+    # For userfetch, drop privileges for the entire fetch call, in
+    # order to handle DISTDIR on NFS with root_squash for bug 601252.
+    if _want_userfetch(mysettings):
+        _drop_privs_userfetch(mysettings)
+
+    return fetch(
+        fetchme,
+        mysettings,
+        listonly=listonly,
+        fetchonly=fetchonly,
+        allow_missing_digests=False,
+        digests=dist_digests,
+    )
+
+
 def _check_temp_dir(settings):
     if "PORTAGE_TMPDIR" not in settings or not os.path.isdir(
         settings["PORTAGE_TMPDIR"]

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     3b975f9b28d1aa6e40c93431086669ad23f6f460
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Mon Sep  4 15:37:24 2023 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sun Dec 10 22:01:48 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=3b975f9b

ebuild: refactor flushing vdb keys

Refactor how we flush these VDB keys in build-info/ to make it easier
to implement bug #913628: it was a pain with the forced-append of a newline
even if we might tamper with the contents later on, so just postpone flushing
to disk until the end. It saves some repetition too, so double win.

Bug: https://bugs.gentoo.org/913628
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 42 ++++++++++++++--------------------
 1 file changed, 17 insertions(+), 25 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 8c7a06b3e2..346c989acc 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2487,8 +2487,8 @@ def _post_src_install_write_metadata(settings):
     """
 
     eapi_attrs = _get_eapi_attrs(settings.configdict["pkg"]["EAPI"])
-
     build_info_dir = os.path.join(settings["PORTAGE_BUILDDIR"], "build-info")
+    metadata_buffer = {}
 
     metadata_keys = ["IUSE"]
     if eapi_attrs.iuse_effective:
@@ -2497,12 +2497,12 @@ def _post_src_install_write_metadata(settings):
     for k in metadata_keys:
         v = settings.configdict["pkg"].get(k)
         if v is not None:
-            write_atomic(os.path.join(build_info_dir, k), v + "\n")
+            metadata_buffer[k] = v
 
     for k in ("CHOST",):
         v = settings.get(k)
         if v is not None:
-            write_atomic(os.path.join(build_info_dir, k), v + "\n")
+            metadata_buffer[k] = v
 
     with open(
         _unicode_encode(
@@ -2542,17 +2542,7 @@ def _post_src_install_write_metadata(settings):
             except OSError:
                 pass
             continue
-        with open(
-            _unicode_encode(
-                os.path.join(build_info_dir, k),
-                encoding=_encodings["fs"],
-                errors="strict",
-            ),
-            mode="w",
-            encoding=_encodings["repo.content"],
-            errors="strict",
-        ) as f:
-            f.write(f"{v}\n")
+        metadata_buffer[k] = v
 
     if eapi_attrs.slot_operator:
         deps = evaluate_slot_operator_equal_deps(settings, use, QueryCommand.get_db())
@@ -2564,18 +2554,20 @@ def _post_src_install_write_metadata(settings):
                 except OSError:
                     pass
                 continue
-            with open(
-                _unicode_encode(
-                    os.path.join(build_info_dir, k),
-                    encoding=_encodings["fs"],
-                    errors="strict",
-                ),
-                mode="w",
-                encoding=_encodings["repo.content"],
-                errors="strict",
-            ) as f:
-                f.write(f"{v}\n")
 
+            metadata_buffer[k] = v
+
+    for k, v in metadata_buffer.items():
+        with open(
+            _unicode_encode(
+                os.path.join(build_info_dir, k),
+                encoding=_encodings["fs"],
+                errors="strict",
+            ),
+            mode="w",
+            encoding=_encodings["repo.content"],
+        ) as f:
+            f.write(f"{v}\n")
 
 def _preinst_bsdflags(mysettings):
     if bsd_chflags:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     e26bd1dfab6f31fb2b176b97ace56a7234d35b4d
Author:     Eli Schwartz <eschwartz93 <AT> gmail <DOT> com>
AuthorDate: Wed Dec 20 05:54:57 2023 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Wed Dec 20 14:04:12 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=e26bd1df

ebuild: use generic variable for QA notices for command not found

calling this "bash_command_not_found" is overly specific. Just because
bash is used to detect the case doesn't mean it's specific to bash. And
really we should extend this find command_not_found for other
environments too.

Signed-off-by: Eli Schwartz <eschwartz93 <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 956f8c0489..d10b157b68 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2270,7 +2270,7 @@ def _check_build_log(mysettings, out=None):
         f = gzip.GzipFile(filename="", mode="rb", fileobj=f)
 
     am_maintainer_mode = []
-    bash_command_not_found = []
+    command_not_found = []
     bash_command_not_found_re = re.compile(
         r"(.*): line (\d*): (.*): command not found$"
     )
@@ -2382,7 +2382,7 @@ def _check_build_log(mysettings, out=None):
                 bash_command_not_found_re.match(line) is not None
                 and command_not_found_exclude_re.search(line) is None
             ):
-                bash_command_not_found.append(line.rstrip("\n"))
+                command_not_found.append(line.rstrip("\n"))
 
             if helper_missing_file_re.match(line) is not None:
                 helper_missing_file.append(line.rstrip("\n"))
@@ -2444,10 +2444,10 @@ def _check_build_log(mysettings, out=None):
         )
         _eqawarn(msg)
 
-    if bash_command_not_found:
+    if command_not_found:
         msg = [_("QA Notice: command not found:")]
         msg.append("")
-        msg.extend("\t" + line for line in bash_command_not_found)
+        msg.extend("\t" + line for line in command_not_found)
         _eqawarn(msg)
 
     if helper_missing_file:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     a2409ba65d38d2eb8cb3923733d9062e3b4da1ad
Author:     Eli Schwartz <eschwartz93 <AT> gmail <DOT> com>
AuthorDate: Wed Dec 20 05:56:53 2023 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Wed Dec 20 14:04:12 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=a2409ba6

ebuild: command not found QA: detect in dash as well

Bug: https://bugs.gentoo.org/822033
Signed-off-by: Eli Schwartz <eschwartz93 <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index d10b157b68..e651f28d06 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2274,6 +2274,7 @@ def _check_build_log(mysettings, out=None):
     bash_command_not_found_re = re.compile(
         r"(.*): line (\d*): (.*): command not found$"
     )
+    dash_command_not_found_re = re.compile(r"(.*): (\d+): (.*): not found$")
     command_not_found_exclude_re = re.compile(r"/configure: line ")
     helper_missing_file = []
     helper_missing_file_re = re.compile(r"^!!! (do|new).*: .* does not exist$")
@@ -2384,6 +2385,12 @@ def _check_build_log(mysettings, out=None):
             ):
                 command_not_found.append(line.rstrip("\n"))
 
+            if (
+                dash_command_not_found_re.match(line) is not None
+                and command_not_found_exclude_re.search(line) is None
+            ):
+                command_not_found.append(line.rstrip("\n"))
+
             if helper_missing_file_re.match(line) is not None:
                 helper_missing_file.append(line.rstrip("\n"))
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     b39e1edca0ee73f29b5513ab30f00da14d1291bc
Author:     Eli Schwartz <eschwartz93 <AT> gmail <DOT> com>
AuthorDate: Wed Dec 20 06:36:38 2023 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Wed Dec 20 14:04:12 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=b39e1edc

ebuild: command not found QA: don't skip QA warnings that come from ./configure

Down through the twisting trail of history, we find a 2008 bug report
and commit 930bbbf31c10265b27825426f1eff6d7f17395e6, which disabled
detecting "command not found" in ./configure scripts with the following
rationale:

> This is actually a false positive. I'll fix it to filter out the ones that are generated by configure scripts.

As far as I can tell it wasn't a false positive though. Looking up the
source code for that old project reveals no indication that it can
misfire in "expected" ways. More generally, this obscures real issues in
configure scripts -- why should configure scripts be special here? --
which have *zero* guarantee of resulting in failure. Anecdotally, they
usually do not.

Whatever the underlying issue is, it's *definitely* an upstream bug and
*almost certainly* indicates that there may be situations where the
project miscompiles against expectations.

Bug: https://bugs.gentoo.org/245716
Signed-off-by: Eli Schwartz <eschwartz93 <AT> gmail.com>
Closes: https://github.com/gentoo/portage/pull/1210
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index e651f28d06..b10bbaf30d 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2275,7 +2275,6 @@ def _check_build_log(mysettings, out=None):
         r"(.*): line (\d*): (.*): command not found$"
     )
     dash_command_not_found_re = re.compile(r"(.*): (\d+): (.*): not found$")
-    command_not_found_exclude_re = re.compile(r"/configure: line ")
     helper_missing_file = []
     helper_missing_file_re = re.compile(r"^!!! (do|new).*: .* does not exist$")
 
@@ -2379,16 +2378,10 @@ def _check_build_log(mysettings, out=None):
             ):
                 am_maintainer_mode.append(line.rstrip("\n"))
 
-            if (
-                bash_command_not_found_re.match(line) is not None
-                and command_not_found_exclude_re.search(line) is None
-            ):
+            if bash_command_not_found_re.match(line) is not None:
                 command_not_found.append(line.rstrip("\n"))
 
-            if (
-                dash_command_not_found_re.match(line) is not None
-                and command_not_found_exclude_re.search(line) is None
-            ):
+            if dash_command_not_found_re.match(line) is not None:
                 command_not_found.append(line.rstrip("\n"))
 
             if helper_missing_file_re.match(line) is not None:

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     72f41d07b5396195a98691fafeb3e5b207a76564
Author:     Raul E Rangel <rrangel <AT> chromium <DOT> org>
AuthorDate: Thu Mar 21 20:29:59 2024 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Mar 24 22:18:34 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=72f41d07

Reapply "config: Don't directly modify FEATURES"

This reverts commit b150419d28bd7afb98404a829c639584d34efc03.

It turns out we need to keep the open coded version to avoid creating
a persistent setting.
See https://github.com/gentoo/portage/pull/1098#issuecomment-1761638611

This change just adds a sorted() so we get deterministic ordering.

Bug: https://bugs.gentoo.org/914441
Signed-off-by: Raul E Rangel <rrangel <AT> chromium.org>
Closes: https://github.com/gentoo/portage/pull/1312
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/config.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index bafdc55a08..67fd1bb18d 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -2206,7 +2206,9 @@ class config:
                 # "test" is in IUSE and USE=test is masked, so execution
                 # of src_test() probably is not reliable. Therefore,
                 # temporarily disable FEATURES=test just for this package.
-                self["FEATURES"] = " ".join(x for x in self.features if x != "test")
+                self["FEATURES"] = " ".join(
+                    x for x in sorted(self.features) if x != "test"
+                )
 
         # Allow _* flags from USE_EXPAND wildcards to pass through here.
         use.difference_update(

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     11c51d7c78f48d4c9842e0d475c26b7068f18c3e
Author:     Eli Schwartz <eschwartz93 <AT> gmail <DOT> com>
AuthorDate: Mon Jun 17 00:12:29 2024 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Mon Jun 17 00:20:41 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=11c51d7c

ebuild: fix maintainer mode checks to work with modern autotools

Modern autotools does not use the --run argument to "missing", so the
check essentially never ever ever ever fired anywhere.

The GNU "missing" script is actually allowed to be run by any software
at all, so checking for "missing --run" was always wrong. Indeed, there
are some packages which use it for their own purposes and added
suppressions for this FP.

The correct solution really is to check for *maintainer mode* by
checking whether *maintainer* programs are run (via "missing"). This
also means we get to check for specific programs which autotools.eclass
would be capable of handling, and don't need to arbitrarily exclude
stuff like help2man (???) which makes things somewhat simpler.

It should be noted that I have observed 3 scenarios for the missing
script to be run via:

- the missing script is surrounded by single quotes, followed by the
  unquoted command
- the missing script is unquoted, and is followed by the unquoted
  command
- legacy: the missing script is unquoted and is followed by --run

We have to handle all three cases via a regex group.

Signed-off-by: Eli Schwartz <eschwartz93 <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 6641cd8341..21ff5a77fd 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2382,10 +2382,10 @@ def _check_build_log(mysettings, out=None):
     #
     # Configuration:
     #  Automake:                   ${SHELL} /var/tmp/portage/dev-libs/yaz-3.0.47/work/yaz-3.0.47/config/missing --run automake-1.10
-    am_maintainer_mode_re = re.compile(r"/missing --run ")
-    am_maintainer_mode_exclude_re = re.compile(
-        r"(/missing --run (autoheader|autotest|help2man|makeinfo)|^\s*Automake:\s)"
+    am_maintainer_mode_re = re.compile(
+        r"/missing( --run|'|) (automake|autoconf|autoheader|aclocal)"
     )
+    am_maintainer_mode_exclude_re = re.compile(r"^\s*Automake:\s")
 
     make_jobserver_re = re.compile(r"g?make\[\d+\]: warning: jobserver unavailable:")
     make_jobserver = []

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     819c86325ce6b966d56d5f4848cf762767ab2bf9
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Mon Jun 17 00:24:52 2024 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Mon Jun 17 00:24:52 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=819c8632

ebuild: fix typo in comment

Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 21ff5a77fd..c7fa1d1e06 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2390,7 +2390,7 @@ def _check_build_log(mysettings, out=None):
     make_jobserver_re = re.compile(r"g?make\[\d+\]: warning: jobserver unavailable:")
     make_jobserver = []
 
-    # we deduplicate these since they is repeated for every setup.py call
+    # we deduplicate these since they are repeated for every setup.py call
     setuptools_warn = set()
     setuptools_warn_re = re.compile(r".*\/setuptools\/.*: .*Warning: (.*)")
     # skip useless version normalization warnings

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Sam James]

commit:     45fceb11558e3363390a4b58ab067603b418773e
Author:     Eli Schwartz <eschwartz <AT> gentoo <DOT> org>
AuthorDate: Mon Jul  8 03:45:05 2024 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Jul 19 05:41:19 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=45fceb11

ebuild: try to make maintainer mode checks a bit less prone to FP

We sometimes get weird results. For example, in binutils/gdb, we get:

```
 * QA Notice: Automake "maintainer mode" detected:
 *
 * 	checking for aclocal... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing aclocal-1.15
 * 	checking for autoconf... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing autoconf
 * 	checking for autoheader... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing autoheader
 * 	checking for aclocal... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing aclocal-1.15
 * 	checking for autoconf... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing autoconf
 * 	checking for autoheader... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing autoheader
 ```

 The current pattern looks for the definition of the "missing" script
 followed by known autotools commands. With a bit more effort, we can
 hopefully match the actual `make` commands that get produced as
 maintainer targets. As far as I can tell, they always include an
 attempt to cd to the project root and then run the maintainer target.

Signed-off-by: Eli Schwartz <eschwartz <AT> gentoo.org>
Closes: https://github.com/gentoo/portage/pull/1359
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index c7fa1d1e06..403836b80b 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2383,7 +2383,7 @@ def _check_build_log(mysettings, out=None):
     # Configuration:
     #  Automake:                   ${SHELL} /var/tmp/portage/dev-libs/yaz-3.0.47/work/yaz-3.0.47/config/missing --run automake-1.10
     am_maintainer_mode_re = re.compile(
-        r"/missing( --run|'|) (automake|autoconf|autoheader|aclocal)"
+        r"\bcd\b.*&&.*/missing( --run|'|) (automake|autoconf|autoheader|aclocal)"
     )
     am_maintainer_mode_exclude_re = re.compile(r"^\s*Automake:\s")
 

[gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/

[Zac Medico]

commit:     9e6451c88e3da11e0eb7b0bd6b1497c5ca4fb67f
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Aug  6 04:48:26 2024 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed Aug  7 14:39:25 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=9e6451c8

doebuild.spawn: Skip socks5 proxy for "depend" phase

Skip the socks5 proxy for the "depend" phase. It should not be needed
because we only allow bash builtin commands during this phase.

Since the socks5 proxy requires portage's event loop to be explictly
closed before exit, skipping it will allow programs like eclean-dist
to avoid the need to explicitly close portage's event loop before exit.

Bug: https://bugs.gentoo.org/937384
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 403836b80b..b5fb46df70 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2009,7 +2009,7 @@ def spawn(
 
         if (
             not networked
-            and mysettings.get("EBUILD_PHASE") != "nofetch"
+            and mysettings.get("EBUILD_PHASE") not in ("depend", "nofetch")
             and ("network-sandbox-proxy" in features or "distcc" in features)
         ):
             # Provide a SOCKS5-over-UNIX-socket proxy to escape sandbox