* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-10-10 7:58 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-10-10 7:58 UTC (permalink / raw
To: gentoo-commits
commit: 1fc6cae0b2de181df5ba4c5b3f21b06604555387
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Wed Oct 10 07:52:10 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed Oct 10 07:57:38 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=1fc6cae0
doebuild: skip timestamp check for deleted distfiles (bug 668206)
Avoid behavior like this when a distfile has been removed:
>>> Existing ${T}/environment for 'chromium-70.0.3538.45' will be sourced.
>>> Run 'clean' to start with a fresh environment.
>>> Checking chromium-70.0.3538.45.tar.xz's mtime...
>>> Timestamp of chromium-70.0.3538.45.tar.xz has changed; recreating WORKDIR...
Bug: https://bugs.gentoo.org/668206
Reported-by: Michał Górny <mgorny <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 621fe7360..9706de422 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -934,10 +934,10 @@ def doebuild(myebuild, mydo, _unused=DeprecationWarning, settings=None, debug=0,
x_st = os.stat(os.path.join(
mysettings["DISTDIR"], x))
except OSError:
- # file not fetched yet
+ # file deleted
x_st = None
- if x_st is None or x_st.st_mtime > workdir_st.st_mtime:
+ if x_st is not None and x_st.st_mtime > workdir_st.st_mtime:
writemsg_stdout(_(">>> Timestamp of "
"%s has changed; recreating WORKDIR...\n") % x)
newstuff = True
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2024-08-07 14:39 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2024-08-07 14:39 UTC (permalink / raw
To: gentoo-commits
commit: 9e6451c88e3da11e0eb7b0bd6b1497c5ca4fb67f
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 6 04:48:26 2024 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed Aug 7 14:39:25 2024 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=9e6451c8
doebuild.spawn: Skip socks5 proxy for "depend" phase
Skip the socks5 proxy for the "depend" phase. It should not be needed
because we only allow bash builtin commands during this phase.
Since the socks5 proxy requires portage's event loop to be explictly
closed before exit, skipping it will allow programs like eclean-dist
to avoid the need to explicitly close portage's event loop before exit.
Bug: https://bugs.gentoo.org/937384
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 403836b80b..b5fb46df70 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2009,7 +2009,7 @@ def spawn(
if (
not networked
- and mysettings.get("EBUILD_PHASE") != "nofetch"
+ and mysettings.get("EBUILD_PHASE") not in ("depend", "nofetch")
and ("network-sandbox-proxy" in features or "distcc" in features)
):
# Provide a SOCKS5-over-UNIX-socket proxy to escape sandbox
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2024-07-19 5:41 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2024-07-19 5:41 UTC (permalink / raw
To: gentoo-commits
commit: 45fceb11558e3363390a4b58ab067603b418773e
Author: Eli Schwartz <eschwartz <AT> gentoo <DOT> org>
AuthorDate: Mon Jul 8 03:45:05 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Jul 19 05:41:19 2024 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=45fceb11
ebuild: try to make maintainer mode checks a bit less prone to FP
We sometimes get weird results. For example, in binutils/gdb, we get:
```
* QA Notice: Automake "maintainer mode" detected:
*
* checking for aclocal... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing aclocal-1.15
* checking for autoconf... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing autoconf
* checking for autoheader... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing autoheader
* checking for aclocal... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing aclocal-1.15
* checking for autoconf... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing autoconf
* checking for autoheader... ${SHELL} /var/tmp/portage/dev-debug/gdb-15.1/work/gdb-15.1/missing autoheader
```
The current pattern looks for the definition of the "missing" script
followed by known autotools commands. With a bit more effort, we can
hopefully match the actual `make` commands that get produced as
maintainer targets. As far as I can tell, they always include an
attempt to cd to the project root and then run the maintainer target.
Signed-off-by: Eli Schwartz <eschwartz <AT> gentoo.org>
Closes: https://github.com/gentoo/portage/pull/1359
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index c7fa1d1e06..403836b80b 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2383,7 +2383,7 @@ def _check_build_log(mysettings, out=None):
# Configuration:
# Automake: ${SHELL} /var/tmp/portage/dev-libs/yaz-3.0.47/work/yaz-3.0.47/config/missing --run automake-1.10
am_maintainer_mode_re = re.compile(
- r"/missing( --run|'|) (automake|autoconf|autoheader|aclocal)"
+ r"\bcd\b.*&&.*/missing( --run|'|) (automake|autoconf|autoheader|aclocal)"
)
am_maintainer_mode_exclude_re = re.compile(r"^\s*Automake:\s")
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2024-06-17 0:25 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2024-06-17 0:25 UTC (permalink / raw
To: gentoo-commits
commit: 819c86325ce6b966d56d5f4848cf762767ab2bf9
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Mon Jun 17 00:24:52 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Mon Jun 17 00:24:52 2024 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=819c8632
ebuild: fix typo in comment
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 21ff5a77fd..c7fa1d1e06 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2390,7 +2390,7 @@ def _check_build_log(mysettings, out=None):
make_jobserver_re = re.compile(r"g?make\[\d+\]: warning: jobserver unavailable:")
make_jobserver = []
- # we deduplicate these since they is repeated for every setup.py call
+ # we deduplicate these since they are repeated for every setup.py call
setuptools_warn = set()
setuptools_warn_re = re.compile(r".*\/setuptools\/.*: .*Warning: (.*)")
# skip useless version normalization warnings
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2024-06-17 0:20 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2024-06-17 0:20 UTC (permalink / raw
To: gentoo-commits
commit: 11c51d7c78f48d4c9842e0d475c26b7068f18c3e
Author: Eli Schwartz <eschwartz93 <AT> gmail <DOT> com>
AuthorDate: Mon Jun 17 00:12:29 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Mon Jun 17 00:20:41 2024 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=11c51d7c
ebuild: fix maintainer mode checks to work with modern autotools
Modern autotools does not use the --run argument to "missing", so the
check essentially never ever ever ever fired anywhere.
The GNU "missing" script is actually allowed to be run by any software
at all, so checking for "missing --run" was always wrong. Indeed, there
are some packages which use it for their own purposes and added
suppressions for this FP.
The correct solution really is to check for *maintainer mode* by
checking whether *maintainer* programs are run (via "missing"). This
also means we get to check for specific programs which autotools.eclass
would be capable of handling, and don't need to arbitrarily exclude
stuff like help2man (???) which makes things somewhat simpler.
It should be noted that I have observed 3 scenarios for the missing
script to be run via:
- the missing script is surrounded by single quotes, followed by the
unquoted command
- the missing script is unquoted, and is followed by the unquoted
command
- legacy: the missing script is unquoted and is followed by --run
We have to handle all three cases via a regex group.
Signed-off-by: Eli Schwartz <eschwartz93 <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 6641cd8341..21ff5a77fd 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2382,10 +2382,10 @@ def _check_build_log(mysettings, out=None):
#
# Configuration:
# Automake: ${SHELL} /var/tmp/portage/dev-libs/yaz-3.0.47/work/yaz-3.0.47/config/missing --run automake-1.10
- am_maintainer_mode_re = re.compile(r"/missing --run ")
- am_maintainer_mode_exclude_re = re.compile(
- r"(/missing --run (autoheader|autotest|help2man|makeinfo)|^\s*Automake:\s)"
+ am_maintainer_mode_re = re.compile(
+ r"/missing( --run|'|) (automake|autoconf|autoheader|aclocal)"
)
+ am_maintainer_mode_exclude_re = re.compile(r"^\s*Automake:\s")
make_jobserver_re = re.compile(r"g?make\[\d+\]: warning: jobserver unavailable:")
make_jobserver = []
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2024-03-24 22:20 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2024-03-24 22:20 UTC (permalink / raw
To: gentoo-commits
commit: 72f41d07b5396195a98691fafeb3e5b207a76564
Author: Raul E Rangel <rrangel <AT> chromium <DOT> org>
AuthorDate: Thu Mar 21 20:29:59 2024 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Mar 24 22:18:34 2024 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=72f41d07
Reapply "config: Don't directly modify FEATURES"
This reverts commit b150419d28bd7afb98404a829c639584d34efc03.
It turns out we need to keep the open coded version to avoid creating
a persistent setting.
See https://github.com/gentoo/portage/pull/1098#issuecomment-1761638611
This change just adds a sorted() so we get deterministic ordering.
Bug: https://bugs.gentoo.org/914441
Signed-off-by: Raul E Rangel <rrangel <AT> chromium.org>
Closes: https://github.com/gentoo/portage/pull/1312
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index bafdc55a08..67fd1bb18d 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -2206,7 +2206,9 @@ class config:
# "test" is in IUSE and USE=test is masked, so execution
# of src_test() probably is not reliable. Therefore,
# temporarily disable FEATURES=test just for this package.
- self["FEATURES"] = " ".join(x for x in self.features if x != "test")
+ self["FEATURES"] = " ".join(
+ x for x in sorted(self.features) if x != "test"
+ )
# Allow _* flags from USE_EXPAND wildcards to pass through here.
use.difference_update(
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-12-20 14:04 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2023-12-20 14:04 UTC (permalink / raw
To: gentoo-commits
commit: e26bd1dfab6f31fb2b176b97ace56a7234d35b4d
Author: Eli Schwartz <eschwartz93 <AT> gmail <DOT> com>
AuthorDate: Wed Dec 20 05:54:57 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Wed Dec 20 14:04:12 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e26bd1df
ebuild: use generic variable for QA notices for command not found
calling this "bash_command_not_found" is overly specific. Just because
bash is used to detect the case doesn't mean it's specific to bash. And
really we should extend this find command_not_found for other
environments too.
Signed-off-by: Eli Schwartz <eschwartz93 <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 956f8c0489..d10b157b68 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2270,7 +2270,7 @@ def _check_build_log(mysettings, out=None):
f = gzip.GzipFile(filename="", mode="rb", fileobj=f)
am_maintainer_mode = []
- bash_command_not_found = []
+ command_not_found = []
bash_command_not_found_re = re.compile(
r"(.*): line (\d*): (.*): command not found$"
)
@@ -2382,7 +2382,7 @@ def _check_build_log(mysettings, out=None):
bash_command_not_found_re.match(line) is not None
and command_not_found_exclude_re.search(line) is None
):
- bash_command_not_found.append(line.rstrip("\n"))
+ command_not_found.append(line.rstrip("\n"))
if helper_missing_file_re.match(line) is not None:
helper_missing_file.append(line.rstrip("\n"))
@@ -2444,10 +2444,10 @@ def _check_build_log(mysettings, out=None):
)
_eqawarn(msg)
- if bash_command_not_found:
+ if command_not_found:
msg = [_("QA Notice: command not found:")]
msg.append("")
- msg.extend("\t" + line for line in bash_command_not_found)
+ msg.extend("\t" + line for line in command_not_found)
_eqawarn(msg)
if helper_missing_file:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-12-20 14:04 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2023-12-20 14:04 UTC (permalink / raw
To: gentoo-commits
commit: b39e1edca0ee73f29b5513ab30f00da14d1291bc
Author: Eli Schwartz <eschwartz93 <AT> gmail <DOT> com>
AuthorDate: Wed Dec 20 06:36:38 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Wed Dec 20 14:04:12 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=b39e1edc
ebuild: command not found QA: don't skip QA warnings that come from ./configure
Down through the twisting trail of history, we find a 2008 bug report
and commit 930bbbf31c10265b27825426f1eff6d7f17395e6, which disabled
detecting "command not found" in ./configure scripts with the following
rationale:
> This is actually a false positive. I'll fix it to filter out the ones that are generated by configure scripts.
As far as I can tell it wasn't a false positive though. Looking up the
source code for that old project reveals no indication that it can
misfire in "expected" ways. More generally, this obscures real issues in
configure scripts -- why should configure scripts be special here? --
which have *zero* guarantee of resulting in failure. Anecdotally, they
usually do not.
Whatever the underlying issue is, it's *definitely* an upstream bug and
*almost certainly* indicates that there may be situations where the
project miscompiles against expectations.
Bug: https://bugs.gentoo.org/245716
Signed-off-by: Eli Schwartz <eschwartz93 <AT> gmail.com>
Closes: https://github.com/gentoo/portage/pull/1210
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 11 ++---------
1 file changed, 2 insertions(+), 9 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index e651f28d06..b10bbaf30d 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2275,7 +2275,6 @@ def _check_build_log(mysettings, out=None):
r"(.*): line (\d*): (.*): command not found$"
)
dash_command_not_found_re = re.compile(r"(.*): (\d+): (.*): not found$")
- command_not_found_exclude_re = re.compile(r"/configure: line ")
helper_missing_file = []
helper_missing_file_re = re.compile(r"^!!! (do|new).*: .* does not exist$")
@@ -2379,16 +2378,10 @@ def _check_build_log(mysettings, out=None):
):
am_maintainer_mode.append(line.rstrip("\n"))
- if (
- bash_command_not_found_re.match(line) is not None
- and command_not_found_exclude_re.search(line) is None
- ):
+ if bash_command_not_found_re.match(line) is not None:
command_not_found.append(line.rstrip("\n"))
- if (
- dash_command_not_found_re.match(line) is not None
- and command_not_found_exclude_re.search(line) is None
- ):
+ if dash_command_not_found_re.match(line) is not None:
command_not_found.append(line.rstrip("\n"))
if helper_missing_file_re.match(line) is not None:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-12-20 14:04 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2023-12-20 14:04 UTC (permalink / raw
To: gentoo-commits
commit: a2409ba65d38d2eb8cb3923733d9062e3b4da1ad
Author: Eli Schwartz <eschwartz93 <AT> gmail <DOT> com>
AuthorDate: Wed Dec 20 05:56:53 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Wed Dec 20 14:04:12 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=a2409ba6
ebuild: command not found QA: detect in dash as well
Bug: https://bugs.gentoo.org/822033
Signed-off-by: Eli Schwartz <eschwartz93 <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index d10b157b68..e651f28d06 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2274,6 +2274,7 @@ def _check_build_log(mysettings, out=None):
bash_command_not_found_re = re.compile(
r"(.*): line (\d*): (.*): command not found$"
)
+ dash_command_not_found_re = re.compile(r"(.*): (\d+): (.*): not found$")
command_not_found_exclude_re = re.compile(r"/configure: line ")
helper_missing_file = []
helper_missing_file_re = re.compile(r"^!!! (do|new).*: .* does not exist$")
@@ -2384,6 +2385,12 @@ def _check_build_log(mysettings, out=None):
):
command_not_found.append(line.rstrip("\n"))
+ if (
+ dash_command_not_found_re.match(line) is not None
+ and command_not_found_exclude_re.search(line) is None
+ ):
+ command_not_found.append(line.rstrip("\n"))
+
if helper_missing_file_re.match(line) is not None:
helper_missing_file.append(line.rstrip("\n"))
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-12-10 22:01 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2023-12-10 22:01 UTC (permalink / raw
To: gentoo-commits
commit: 3b975f9b28d1aa6e40c93431086669ad23f6f460
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Mon Sep 4 15:37:24 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sun Dec 10 22:01:48 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=3b975f9b
ebuild: refactor flushing vdb keys
Refactor how we flush these VDB keys in build-info/ to make it easier
to implement bug #913628: it was a pain with the forced-append of a newline
even if we might tamper with the contents later on, so just postpone flushing
to disk until the end. It saves some repetition too, so double win.
Bug: https://bugs.gentoo.org/913628
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 42 ++++++++++++++--------------------
1 file changed, 17 insertions(+), 25 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 8c7a06b3e2..346c989acc 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2487,8 +2487,8 @@ def _post_src_install_write_metadata(settings):
"""
eapi_attrs = _get_eapi_attrs(settings.configdict["pkg"]["EAPI"])
-
build_info_dir = os.path.join(settings["PORTAGE_BUILDDIR"], "build-info")
+ metadata_buffer = {}
metadata_keys = ["IUSE"]
if eapi_attrs.iuse_effective:
@@ -2497,12 +2497,12 @@ def _post_src_install_write_metadata(settings):
for k in metadata_keys:
v = settings.configdict["pkg"].get(k)
if v is not None:
- write_atomic(os.path.join(build_info_dir, k), v + "\n")
+ metadata_buffer[k] = v
for k in ("CHOST",):
v = settings.get(k)
if v is not None:
- write_atomic(os.path.join(build_info_dir, k), v + "\n")
+ metadata_buffer[k] = v
with open(
_unicode_encode(
@@ -2542,17 +2542,7 @@ def _post_src_install_write_metadata(settings):
except OSError:
pass
continue
- with open(
- _unicode_encode(
- os.path.join(build_info_dir, k),
- encoding=_encodings["fs"],
- errors="strict",
- ),
- mode="w",
- encoding=_encodings["repo.content"],
- errors="strict",
- ) as f:
- f.write(f"{v}\n")
+ metadata_buffer[k] = v
if eapi_attrs.slot_operator:
deps = evaluate_slot_operator_equal_deps(settings, use, QueryCommand.get_db())
@@ -2564,18 +2554,20 @@ def _post_src_install_write_metadata(settings):
except OSError:
pass
continue
- with open(
- _unicode_encode(
- os.path.join(build_info_dir, k),
- encoding=_encodings["fs"],
- errors="strict",
- ),
- mode="w",
- encoding=_encodings["repo.content"],
- errors="strict",
- ) as f:
- f.write(f"{v}\n")
+ metadata_buffer[k] = v
+
+ for k, v in metadata_buffer.items():
+ with open(
+ _unicode_encode(
+ os.path.join(build_info_dir, k),
+ encoding=_encodings["fs"],
+ errors="strict",
+ ),
+ mode="w",
+ encoding=_encodings["repo.content"],
+ ) as f:
+ f.write(f"{v}\n")
def _preinst_bsdflags(mysettings):
if bsd_chflags:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-10-22 22:46 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2023-10-22 22:46 UTC (permalink / raw
To: gentoo-commits
commit: e9d6782ca18b2f30d97b4859419bf7da4da01bab
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Oct 22 21:24:47 2023 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Oct 22 22:45:02 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e9d6782c
doebuild: Make _fetch_subprocess compatible with spawn
Before it failed like this with the multiprocessing spawn
start method:
AttributeError: Can't pickle local object 'doebuild.<locals>._fetch_subprocess'
Bug: https://bugs.gentoo.org/916139
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 32 +++++++++++++++++---------------
1 file changed, 17 insertions(+), 15 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index b967fc061f..ba166173a8 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1305,21 +1305,6 @@ def doebuild(
if mf is not None:
dist_digests = mf.getTypeDigests("DIST")
- def _fetch_subprocess(fetchme, mysettings, listonly, dist_digests):
- # For userfetch, drop privileges for the entire fetch call, in
- # order to handle DISTDIR on NFS with root_squash for bug 601252.
- if _want_userfetch(mysettings):
- _drop_privs_userfetch(mysettings)
-
- return fetch(
- fetchme,
- mysettings,
- listonly=listonly,
- fetchonly=fetchonly,
- allow_missing_digests=False,
- digests=dist_digests,
- )
-
loop = asyncio._safe_loop()
if loop.is_running():
# Called by EbuildFetchonly for emerge --pretend --fetchonly.
@@ -1340,6 +1325,7 @@ def doebuild(
mysettings,
listonly,
dist_digests,
+ fetchonly,
)
)
if not success:
@@ -1589,6 +1575,22 @@ def doebuild(
portage._doebuild_manifest_exempt_depend -= 1
+def _fetch_subprocess(fetchme, mysettings, listonly, dist_digests, fetchonly):
+ # For userfetch, drop privileges for the entire fetch call, in
+ # order to handle DISTDIR on NFS with root_squash for bug 601252.
+ if _want_userfetch(mysettings):
+ _drop_privs_userfetch(mysettings)
+
+ return fetch(
+ fetchme,
+ mysettings,
+ listonly=listonly,
+ fetchonly=fetchonly,
+ allow_missing_digests=False,
+ digests=dist_digests,
+ )
+
+
def _check_temp_dir(settings):
if "PORTAGE_TMPDIR" not in settings or not os.path.isdir(
settings["PORTAGE_TMPDIR"]
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-10-16 5:15 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2023-10-16 5:15 UTC (permalink / raw
To: gentoo-commits
commit: ea1c12ac91faeb25b30d364afa1764506b7a1535
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 16 04:48:10 2023 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 16 05:14:08 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=ea1c12ac
_post_src_install_uid_fix: Pass unicode to write_atomic for utf8_mode
Bug: https://bugs.gentoo.org/915837
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 5780c2b0b3..b967fc061f 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2777,7 +2777,9 @@ def _post_src_install_uid_fix(mysettings, out):
# a normal write might fail due to file permission
# settings on some operating systems such as HP-UX
write_atomic(
- _unicode_encode(
+ fpath
+ if portage.utf8_mode
+ else _unicode_encode(
fpath, encoding=_encodings["merge"], errors="strict"
),
new_contents,
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-10-13 10:33 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2023-10-13 10:33 UTC (permalink / raw
To: gentoo-commits
commit: b150419d28bd7afb98404a829c639584d34efc03
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri Oct 13 10:32:26 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Oct 13 10:33:13 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=b150419d
Revert "config: Don't directly modify FEATURES"
This reverts commit 8c2d81eb205bf05445eca616818681e047639eb5.
I get a bunch of lost USE=test with this:
```
$ emerge -p -uvDU @world
[...]
[ebuild R ] dev-libs/tree-sitter-bash-0.20.4-r1::gentoo USE="-test*" 0 KiB
[ebuild R ] dev-libs/libffi-3.4.4-r1:0/8::gentoo USE="-debug (-exec-static-trampoline) -pax-kernel -static-libs -test*" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild R ] dev-util/pkgconf-2.0.3:0/4::gentoo USE="-test*" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild R ] dev-libs/libpipeline-1.5.7::gentoo USE="-test*" 0 KiB
[ebuild R ] dev-util/fq-0.8.0::gentoo USE="-test*" 0 KiB
[...]
```
Bug: https://bugs.gentoo.org/914441
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 049c5fa169..f51886c33e 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -2204,7 +2204,7 @@ class config:
# "test" is in IUSE and USE=test is masked, so execution
# of src_test() probably is not reliable. Therefore,
# temporarily disable FEATURES=test just for this package.
- self.features.remove("test")
+ self["FEATURES"] = " ".join(x for x in self.features if x != "test")
# Allow _* flags from USE_EXPAND wildcards to pass through here.
use.difference_update(
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-10-13 10:19 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2023-10-13 10:19 UTC (permalink / raw
To: gentoo-commits
commit: 8c2d81eb205bf05445eca616818681e047639eb5
Author: Raul E Rangel <rrangel <AT> chromium <DOT> org>
AuthorDate: Tue Sep 19 20:43:21 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Oct 13 10:19:00 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=8c2d81eb
config: Don't directly modify FEATURES
Iterating on self.features returns items in a non-deterministic order,
it also leaves self.features out of sync. If we instead use the `remove`
method, the values get sorted and synchronized correctly.
Bug: https://bugs.gentoo.org/914441
Signed-off-by: Raul E Rangel <rrangel <AT> chromium.org>
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index e07d27e8fc..3ee7fd65ed 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -2204,7 +2204,7 @@ class config:
# "test" is in IUSE and USE=test is masked, so execution
# of src_test() probably is not reliable. Therefore,
# temporarily disable FEATURES=test just for this package.
- self["FEATURES"] = " ".join(x for x in self.features if x != "test")
+ self.features.remove("test")
# Allow _* flags from USE_EXPAND wildcards to pass through here.
use.difference_update(
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-10-13 10:19 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2023-10-13 10:19 UTC (permalink / raw
To: gentoo-commits
commit: c16ff6913b3d0a4d4ea0f6ee811ece541865bd8f
Author: Raul E Rangel <rrangel <AT> chromium <DOT> org>
AuthorDate: Wed Oct 11 18:42:34 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Oct 13 10:19:01 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=c16ff691
config: Sort USE_EXPAND variables
Without the sort, we end up extending the USE_EXPAND variables with
a non deterministic ordering. This makes binary package creation non
hermetic.
i.e.,
```
-declare -x PYTHON_TARGETS="python3_8 python3_7 python3_6 python3_10 python3_9"
+declare -x PYTHON_TARGETS="python3_8 python3_9 python3_10 python3_7 python3_6"
```
Assuming `PYTHON_TARGETS=python3_8` in make.conf, after this change we
get:
```
declare -x PYTHON_TARGETS="python3_8 python3_10 python3_6 python3_7 python3_9"
```
Ideally we would completely sort the USE_EXPAND variables, but the
LINGUAS variable appears to need a very specific ordering.
Bug: https://bugs.gentoo.org/914441
Signed-off-by: Raul E Rangel <rrangel <AT> chromium.org>
Closes: https://github.com/gentoo/portage/pull/1098
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 3ee7fd65ed..049c5fa169 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1746,7 +1746,7 @@ class config:
# Preserve the order of var_split because it can matter for things
# like LINGUAS.
var_split = [x for x in var_split if x in expand_flags]
- var_split.extend(expand_flags.difference(var_split))
+ var_split.extend(sorted(expand_flags.difference(var_split)))
has_wildcard = "*" in expand_flags
if has_wildcard:
var_split = [x for x in var_split if x != "*"]
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-08-19 15:05 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2023-08-19 15:05 UTC (permalink / raw
To: gentoo-commits
commit: 87d0a7447f1464157f18a8ba0991719777003325
Author: Oskari Pirhonen <xxc3ncoredxx <AT> gmail <DOT> com>
AuthorDate: Sun Aug 13 03:11:15 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sat Aug 19 15:05:19 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=87d0a744
doebuild: gate -Oline behind MAKEFLAGS check
Add a check to see if MAKEFLAGS is set (alongside the existing MAKEOPTS
and GNUMAKEFLAGS checks) before enabling output synchronization.
This is only tangentially related to the bug below via being a (likely
rare) case of MAKEFLAGS being set in the wild.
Bug: https://bugs.gentoo.org/909009
Signed-off-by: Oskari Pirhonen <xxc3ncoredxx <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 8b65a7862f..4d4e674c4b 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -589,7 +589,7 @@ def doebuild_environment(
nproc = get_cpu_count()
if nproc:
mysettings["MAKEOPTS"] = "-j%d" % (nproc)
- if "GNUMAKEFLAGS" not in mysettings:
+ if "GNUMAKEFLAGS" not in mysettings and "MAKEFLAGS" not in mysettings:
mysettings["GNUMAKEFLAGS"] = "--output-sync=line"
if not eapi_exports_KV(eapi):
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-06-14 19:23 Mike Gilbert
0 siblings, 0 replies; 93+ messages in thread
From: Mike Gilbert @ 2023-06-14 19:23 UTC (permalink / raw
To: gentoo-commits
commit: 37b108a7a4630583921484c0b5f513a7e384d851
Author: Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Tue Jun 6 18:03:50 2023 +0000
Commit: Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Wed Jun 14 19:21:01 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=37b108a7
doebuild: do not rely on os.access() for PORTAGE_TMPDIR write check
Calling os.access() on ${PORTAGE_TMPDIR}/portage will not trigger any
automount that the user may have configured there.
Instead, just try to create a file and catch PermissionError.
Bug: https://bugs.gentoo.org/890812
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index d30c6b8f0..8b65a7862 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1608,7 +1608,22 @@ def _check_temp_dir(settings):
# for those people.
checkdir = first_existing(os.path.join(settings["PORTAGE_TMPDIR"], "portage"))
- if not os.access(checkdir, os.W_OK):
+ try:
+ with tempfile.NamedTemporaryFile(prefix="exectest-", dir=checkdir) as fd:
+ os.chmod(fd.name, 0o755)
+ if not os.access(fd.name, os.X_OK):
+ writemsg(
+ _(
+ "Can not execute files in %s\n"
+ "Likely cause is that you've mounted it with one of the\n"
+ "following mount options: 'noexec', 'user', 'users'\n\n"
+ "Please make sure that portage can execute files in this directory.\n"
+ )
+ % checkdir,
+ noiselevel=-1,
+ )
+ return 1
+ except PermissionError:
writemsg(
_(
"%s is not writable.\n"
@@ -1619,21 +1634,6 @@ def _check_temp_dir(settings):
)
return 1
- with tempfile.NamedTemporaryFile(prefix="exectest-", dir=checkdir) as fd:
- os.chmod(fd.name, 0o755)
- if not os.access(fd.name, os.X_OK):
- writemsg(
- _(
- "Can not execute files in %s\n"
- "Likely cause is that you've mounted it with one of the\n"
- "following mount options: 'noexec', 'user', 'users'\n\n"
- "Please make sure that portage can execute files in this directory.\n"
- )
- % checkdir,
- noiselevel=-1,
- )
- return 1
-
return os.EX_OK
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-06-14 19:23 Mike Gilbert
0 siblings, 0 replies; 93+ messages in thread
From: Mike Gilbert @ 2023-06-14 19:23 UTC (permalink / raw
To: gentoo-commits
commit: f5ae3c4b198f0d5543afbdd36d923422cd451272
Author: Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Tue Jun 6 18:15:36 2023 +0000
Commit: Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Wed Jun 14 19:21:55 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=f5ae3c4b
fetch: do not rely on os.stat() for DISTDIR write test
If DISTDIR is an automount, os.stat() will not trigger it.
Just try to create a file instead.
Bug: https://bugs.gentoo.org/485100
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 17 ++++++-----------
1 file changed, 6 insertions(+), 11 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 0d30cb88e..0ecad162e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -240,20 +240,15 @@ def _ensure_distdir(settings, distdir):
userpriv = portage.data.secpass >= 2 and "userpriv" in settings.features
write_test_file = os.path.join(distdir, ".__portage_test_write__")
- try:
- st = os.stat(distdir)
- except OSError:
- st = None
-
- if st is not None and stat.S_ISDIR(st.st_mode):
- if not (userfetch or userpriv):
- return
- if _userpriv_test_write_file(settings, write_test_file):
- return
+ if _userpriv_test_write_file(settings, write_test_file):
+ return
_userpriv_test_write_file_cache.pop(write_test_file, None)
+
+ already_exists = os.path.isdir(distdir)
+
if ensure_dirs(distdir, gid=dir_gid, mode=dirmode, mask=modemask):
- if st is None:
+ if not already_exists:
# The directory has just been created
# and therefore it must be empty.
return
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-02-17 0:53 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2023-02-17 0:53 UTC (permalink / raw
To: gentoo-commits
commit: 010d688e6bd9bb19460b920896549e67a8cb5004
Author: Oskari Pirhonen <xxc3ncoredxx <AT> gmail <DOT> com>
AuthorDate: Sat Nov 19 07:02:21 2022 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Feb 17 00:52:11 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=010d688e
ebuild/doebuild.py: use -Oline if MAKEOPTS unset
If `MAKEOPTS` is unset in `make.conf`, add
`GNUMAKEFLAGS="--output-sync=line"` to the environment to ensure proper
synchronization of output when doing parallel builds. `GNUMAKEFLAGS` is
used to avoid portability issues from passing the non-standard option
directly to `make`.
Signed-off-by: Oskari Pirhonen <xxc3ncoredxx <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index b72d61c7a..2226812e3 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -592,6 +592,8 @@ def doebuild_environment(
nproc = get_cpu_count()
if nproc:
mysettings["MAKEOPTS"] = "-j%d" % (nproc)
+ if "GNUMAKEFLAGS" not in mysettings:
+ mysettings["GNUMAKEFLAGS"] = "--output-sync=line"
if not eapi_exports_KV(eapi):
# Discard KV for EAPIs that don't support it. Cached KV is restored
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-01-27 8:38 Ulrich Müller
0 siblings, 0 replies; 93+ messages in thread
From: Ulrich Müller @ 2023-01-27 8:38 UTC (permalink / raw
To: gentoo-commits
commit: fe3d9c11ba3460e8c5677ebfe45399e3396ea8d9
Author: Ulrich Müller <ulm <AT> gentoo <DOT> org>
AuthorDate: Fri Jan 27 08:37:21 2023 +0000
Commit: Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Fri Jan 27 08:37:21 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=fe3d9c11
package/ebuild/config.py: Move a comment to make black happy
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index f66bf6485..9ed59eb74 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -3347,8 +3347,8 @@ class config:
if not (src_like_phase and eapi_attrs.broot):
mydict.pop("BROOT", None)
- # Prefix variables are supported beginning with EAPI 3.
if phase == "depend" or (
+ # Prefix variables are supported beginning with EAPI 3.
eapi is not None
and not eapi_supports_prefix(eapi)
):
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2023-01-02 20:45 Mike Gilbert
0 siblings, 0 replies; 93+ messages in thread
From: Mike Gilbert @ 2023-01-02 20:45 UTC (permalink / raw
To: gentoo-commits
commit: 69f2221413852630b2c231774e1f811f350fa3e6
Author: Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Sat Dec 31 22:01:08 2022 +0000
Commit: Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Mon Jan 2 20:35:27 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=69f22214
_doebuild_path: simplify logic used to set PATH
Remove logic to incoporate PREROOTPATH and ROOTPATH. I'm not sure where
PREROOTPATH was ever used, and ROOTPATH has been deprecated in
baselayout for a while.
Remove logic to add hard-coded directories like
${EPREFIX}/{,usr{,/local}/{sbin,bin}. Adding these paths is unnecessary
if env.d or the calling environment have a valid PATH setting.
Add logic to ignore PATH from the calling environment if PATH is set in
env.d. This ensures that packages can update PATH by installing files in
/etc/env.d and this will work without having to restart Portage with an
updated environment.
Bug: https://bugs.gentoo.org/607696
Bug: https://bugs.gentoo.org/693308
Bug: https://bugs.gentoo.org/888543
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 44 +++++++++++-----------------------
1 file changed, 14 insertions(+), 30 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 05336e2aa..d29451efa 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -284,20 +284,8 @@ def _doebuild_path(settings, eapi=None):
if portage_bin_path[0] != portage.const.PORTAGE_BIN_PATH:
# Add a fallback path for restarting failed builds (bug 547086)
portage_bin_path.append(portage.const.PORTAGE_BIN_PATH)
- prerootpath = [x for x in settings.get("PREROOTPATH", "").split(":") if x]
- rootpath = [x for x in settings.get("ROOTPATH", "").split(":") if x]
- rootpath_set = frozenset(rootpath)
- overrides = [
- x for x in settings.get("__PORTAGE_TEST_PATH_OVERRIDE", "").split(":") if x
- ]
-
- prefixes = []
- # settings["EPREFIX"] should take priority over portage.const.EPREFIX
- if portage.const.EPREFIX != settings["EPREFIX"] and settings["ROOT"] == os.sep:
- prefixes.append(settings["EPREFIX"])
- prefixes.append(portage.const.EPREFIX)
- path = overrides
+ path = [x for x in settings.get("__PORTAGE_TEST_PATH_OVERRIDE", "").split(":") if x]
if "xattr" in settings.features:
for x in portage_bin_path:
@@ -317,24 +305,20 @@ def _doebuild_path(settings, eapi=None):
for x in portage_bin_path:
path.append(os.path.join(x, "ebuild-helpers"))
- path.extend(prerootpath)
-
- for prefix in prefixes:
- prefix = prefix if prefix else "/"
- for x in (
- "usr/local/sbin",
- "usr/local/bin",
- "usr/sbin",
- "usr/bin",
- "sbin",
- "bin",
- ):
- # Respect order defined in ROOTPATH
- x_abs = os.path.join(prefix, x)
- if x_abs not in rootpath_set:
- path.append(x_abs)
- path.extend(rootpath)
+ # If PATH is set in env.d, ignore PATH from the calling environment.
+ # This allows packages to update our PATH as they get installed.
+ if "PATH" in settings.configdict["env.d"]:
+ settings.configdict["env"].pop("PATH", None)
+
+ if "PATH" in settings:
+ pathset = set(path)
+ for p in settings["PATH"].split(":"):
+ # Avoid duplicate entries.
+ if p not in pathset:
+ path.append(p)
+ pathset.add(p)
+
settings["PATH"] = ":".join(path)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2022-07-12 23:33 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2022-07-12 23:33 UTC (permalink / raw
To: gentoo-commits
commit: 7cc778fbf3aa7919036f2bb224b118d96c14f31e
Author: Arfrever Frehtes Taifersar Arahesis <arfrever <AT> apache <DOT> org>
AuthorDate: Mon Jul 11 02:54:26 2022 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Tue Jul 12 23:30:05 2022 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=7cc778fb
env-update: handle BROOT vs ROOT distinction for e.g. PATH
When Portage builds packages with a non-trivial ROOT setting, say,
ROOT=/tmp/stage1root, the build dependencies (compilers, tools, etc.)
should be taken from the actual build environment BROOT, say BROOT=/ ,
and that must also be reflected in the environment.
An example situation that leads to problems otherwise is the following:
* We build a clang-based stage1 with CC=clang
* The clang binary exists only in a custom directory pulled into PATH
via env.d.
* ROOT=/tmp/stage1root is initially empty, meaning portage tries to
build with settings from an empty env.d, and suddently cannot find
the clang that perfectly exists in BROOT=/ anymore.
* The build fails.
The patch takes a more versatile approach and separates environment
variables into three categories:
* constructed from ROOT
* constructed from BROOT
* combined from ROOT and BROOT (checks EROOT, then BROOT. If present in both,
only EROOT is used.)
Thanks-to: Andreas K. Hüttel <dilfridge <AT> gentoo.org>
Thanks-to: James Le Cuirot <chewi <AT> gentoo.org>
Signed-off-by: Sam James <sam <AT> gentoo.org>
Closes: https://github.com/gentoo/portage/pull/851
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 77 +++++++++++++++++++++++++++++-------
1 file changed, 62 insertions(+), 15 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 49a906559..2530c9d31 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -485,14 +485,7 @@ class config:
# interaction with the calling environment that might
# lead to unexpected results.
- env_d = (
- getconfig(
- os.path.join(eroot, "etc", "profile.env"),
- tolerant=tolerant,
- expand=False,
- )
- or {}
- )
+ env_d = self._get_env_d(broot=broot, eroot=eroot, tolerant=tolerant)
expand_map = env_d.copy()
self._expand_map = expand_map
@@ -1209,6 +1202,61 @@ class config:
if mycpv:
self.setcpv(mycpv)
+ def _get_env_d(self, broot, eroot, tolerant):
+ broot_only_variables = (
+ "PATH",
+ "PREROOTPATH",
+ "ROOTPATH",
+ )
+ eroot_only_variables = (
+ "CONFIG_PROTECT",
+ "CONFIG_PROTECT_MASK",
+ "INFODIR",
+ "INFOPATH",
+ "MANPATH",
+ "PKG_CONFIG_.*",
+ )
+
+ broot_only_variables_re = re.compile(r"^(%s)$" % "|".join(broot_only_variables))
+ eroot_only_variables_re = re.compile(r"^(%s)$" % "|".join(eroot_only_variables))
+
+ broot_env_d_path = os.path.join(broot or "/", "etc", "profile.env")
+ eroot_env_d_path = os.path.join(eroot or "/", "etc", "profile.env")
+
+ if (
+ os.path.exists(broot_env_d_path)
+ and os.path.exists(eroot_env_d_path)
+ and os.path.samefile(broot_env_d_path, eroot_env_d_path)
+ ):
+ broot_env_d = (
+ getconfig(broot_env_d_path, tolerant=tolerant, expand=False) or {}
+ )
+ eroot_env_d = broot_env_d
+ else:
+ broot_env_d = (
+ getconfig(broot_env_d_path, tolerant=tolerant, expand=False) or {}
+ )
+ eroot_env_d = (
+ getconfig(eroot_env_d_path, tolerant=tolerant, expand=False) or {}
+ )
+
+ env_d = {}
+
+ for k in broot_env_d.keys() | eroot_env_d.keys():
+ if broot_only_variables_re.match(k):
+ if k in broot_env_d:
+ env_d[k] = broot_env_d[k]
+ elif eroot_only_variables_re.match(k):
+ if k in eroot_env_d:
+ env_d[k] = eroot_env_d[k]
+ else:
+ if k in eroot_env_d:
+ env_d[k] = eroot_env_d[k]
+ elif k in broot_env_d:
+ env_d[k] = broot_env_d[k]
+
+ return env_d
+
def _init_iuse(self):
self._iuse_effective = self._calc_iuse_effective()
self._iuse_implicit_match = _iuse_implicit_match_cache(self)
@@ -2640,14 +2688,13 @@ class config:
def reload(self):
"""Reload things like /etc/profile.env that can change during runtime."""
- env_d_filename = os.path.join(self["EROOT"], "etc", "profile.env")
self.configdict["env.d"].clear()
- env_d = getconfig(env_d_filename, tolerant=self._tolerant, expand=False)
- if env_d:
- # env_d will be None if profile.env doesn't exist.
- for k in self._env_d_blacklist:
- env_d.pop(k, None)
- self.configdict["env.d"].update(env_d)
+ env_d = self._get_env_d(
+ broot=self["BROOT"], eroot=self["EROOT"], tolerant=self._tolerant
+ )
+ for k in self._env_d_blacklist:
+ env_d.pop(k, None)
+ self.configdict["env.d"].update(env_d)
def regenerate(self, useonly=0, use_cache=None):
"""
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2022-06-12 18:25 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2022-06-12 18:25 UTC (permalink / raw
To: gentoo-commits
commit: e0c1a5badef219cc2247c0fd6548776e753c073b
Author: Rin Cat (鈴猫) <syu.os <AT> protonmail <DOT> com>
AuthorDate: Sun Jun 12 18:18:07 2022 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sun Jun 12 18:25:33 2022 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e0c1a5ba
Fix variable typo
Signed-off-by: Sheng Yu <syu.os <AT> protonmail.com>
Closes: https://github.com/gentoo/portage/pull/839
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 2 +-
lib/portage/package/ebuild/doebuild.py | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index f35fcdf65..49a906559 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1510,7 +1510,7 @@ class config:
if binpkg_format not in SUPPORTED_GENTOO_BINPKG_FORMATS:
writemsg(
"!!! BINPKG_FORMAT contains invalid or "
- "unsupported format: %s" % binpkg_fotmat,
+ "unsupported format: %s" % binpkg_format,
noiselevel=-1,
)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 2251cdb8d..8ee9f73c6 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -647,7 +647,7 @@ def doebuild_environment(
if binpkg_format not in portage.const.SUPPORTED_GENTOO_BINPKG_FORMATS:
writemsg(
"!!! BINPKG_FORMAT contains invalid or "
- "unsupported format: %s" % binpkg_fotmat,
+ "unsupported format: %s" % binpkg_format,
noiselevel=-1,
)
binpkg_format = "xpak"
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-12-11 2:54 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2021-12-11 2:54 UTC (permalink / raw
To: gentoo-commits
commit: c36f878509108c970907dc391d6562297881b400
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Sat Dec 11 02:47:50 2021 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sat Dec 11 02:54:00 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=c36f8785
lib/portage/package/ebuild/doebuild.py: use 'eerror' for blank PROVIDES message
This means that it's much harder to miss the error.
Bug: https://bugs.gentoo.org/811462
Reported-by: Brian Dolbec <dolsen <AT> gentoo.org>
Thanks-to: Arfrever Frehtes Taifersar Arahesis <Arfrever <AT> Apache.Org>
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 0ff64e275..ac627f555 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -3096,13 +3096,11 @@ def _post_src_install_soname_symlinks(mysettings, out):
f.write(soname_deps.provides)
else:
if check_dyn_libs_inconsistent(image_dir, soname_deps.provides):
- portage.util._writemsg_level(
- colorize(
- "BAD",
- "!!! Error! Installing dynamic libraries (.so) with blank PROVIDES!",
- ),
- noiselevel=-1,
- level=logging.ERROR,
+ eerror(
+ "Error! Installing dynamic libraries (.so) with blank PROVIDES!",
+ phase="install",
+ key=mysettings.mycpv,
+ out=out,
)
if unrecognized_elf_files:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-12-11 2:54 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2021-12-11 2:54 UTC (permalink / raw
To: gentoo-commits
commit: 80a90acd3da4f24df4d6f66b070c785616c1ee34
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri Dec 10 22:50:05 2021 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sat Dec 11 02:53:59 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=80a90acd
lib/portage/package/ebuild/doebuild.py: fix undefined self
Fixes: 98a8a4af2c77477c327b22621b8a0aed6d15f574
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 34720c7c6..0ff64e275 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -3096,7 +3096,7 @@ def _post_src_install_soname_symlinks(mysettings, out):
f.write(soname_deps.provides)
else:
if check_dyn_libs_inconsistent(image_dir, soname_deps.provides):
- self._writemsg_level(
+ portage.util._writemsg_level(
colorize(
"BAD",
"!!! Error! Installing dynamic libraries (.so) with blank PROVIDES!",
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-11-15 8:34 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2021-11-15 8:34 UTC (permalink / raw
To: gentoo-commits
commit: c309328c4e1f6254251d31149ee47b4266d4d70f
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Mon Nov 15 08:31:30 2021 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Mon Nov 15 08:31:30 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=c309328c
package.ebuild.doebuild: ignore setuptools install deprecation
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 7706bf8cb..34720c7c6 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2359,7 +2359,10 @@ def _check_build_log(mysettings, out=None):
setuptools_warn = set()
setuptools_warn_re = re.compile(r".*\/setuptools\/.*: .*Warning: (.*)")
# skip useless version normalization warnings
- setuptools_warn_ignore_re = [re.compile(r"Normalizing .*")]
+ setuptools_warn_ignore_re = [
+ re.compile(r"Normalizing .*"),
+ re.compile(r"setup.py install is deprecated"),
+ ]
def _eerror(lines):
for line in lines:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-11-08 22:37 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-11-08 22:37 UTC (permalink / raw
To: gentoo-commits
commit: 776ede149514db41c9c08276594b43b2682ba1a4
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Nov 5 04:08:19 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Nov 5 04:22:48 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=776ede14
fetch: enable resume for digestgen case
Enable resume for the digestgen case (no digests available), when
the temporary file exceeds PORTAGE_FETCH_RESUME_MIN_SIZE. This
fixes a case which caused the ebuild digest command to skip the
download and fail with a message like this:
!!! File b'/var/cache/distfiles/foo.tar.xz' doesn't exist, can't update Manifest
Bug: https://bugs.gentoo.org/821571
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 8c64362c2..2d3625800 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1485,10 +1485,11 @@ def fetch(
except EnvironmentError:
pass
elif not orig_digests:
- # We don't have a digest, but the file exists. We must
- # assume that it is fully downloaded.
+ # We don't have a digest, and the temporary file exists.
if not force:
- continue
+ # Try to resume this download when full
+ # download has not been explicitly forced.
+ fetched = 1
else:
if (
mydigests[myfile].get("size") is not None
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-10-28 5:00 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2021-10-28 5:00 UTC (permalink / raw
To: gentoo-commits
commit: e7fa8671f71cae3ea18be510f708b3c63877becd
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Thu Oct 28 04:59:39 2021 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Thu Oct 28 05:00:11 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e7fa8671
lib/portage/package/ebuild/doebuild.py: drop unused import
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 96e56b277..7706bf8cb 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -3,7 +3,6 @@
__all__ = ["doebuild", "doebuild_environment", "spawn", "spawnebuild"]
-import glob
import grp
import gzip
import errno
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-10-28 4:52 Sam James
0 siblings, 0 replies; 93+ messages in thread
From: Sam James @ 2021-10-28 4:52 UTC (permalink / raw
To: gentoo-commits
commit: 98a8a4af2c77477c327b22621b8a0aed6d15f574
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Thu Oct 28 04:40:09 2021 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Thu Oct 28 04:52:22 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=98a8a4af
doebuild.py: check for inconsistent PROVIDES/image post-src_install
This is part of a series of fixes for the linked bug (failure
to preserve libraries in some situations).
At the point of installation (even if not merging), we need
to detect inconsistent metadata: PROVIDES should be populated
if we're installing any dynamic libraries. This suggests that
e.g. scanelf malfunctioned or some corruption occurred.
Bug: https://bugs.gentoo.org/811462
Signed-off-by: Sam James <sam <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 9650a8444..96e56b277 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -3,6 +3,7 @@
__all__ = ["doebuild", "doebuild_environment", "spawn", "spawnebuild"]
+import glob
import grp
import gzip
import errno
@@ -118,6 +119,7 @@ from portage.util.futures import asyncio
from portage.util.futures.executor.fork import ForkExecutor
from portage.util.path import first_existing
from portage.util.socks5 import get_socks5_proxy
+from portage.util._dyn_libs.dyn_libs import check_dyn_libs_inconsistent
from portage.versions import _pkgsplit
from _emerge.BinpkgEnvExtractor import BinpkgEnvExtractor
from _emerge.EbuildBuildDir import EbuildBuildDir
@@ -126,7 +128,6 @@ from _emerge.EbuildSpawnProcess import EbuildSpawnProcess
from _emerge.Package import Package
from _emerge.RootConfig import RootConfig
-
_unsandboxed_phases = frozenset(
[
"clean",
@@ -3091,6 +3092,16 @@ def _post_src_install_soname_symlinks(mysettings, out):
errors="strict",
) as f:
f.write(soname_deps.provides)
+ else:
+ if check_dyn_libs_inconsistent(image_dir, soname_deps.provides):
+ self._writemsg_level(
+ colorize(
+ "BAD",
+ "!!! Error! Installing dynamic libraries (.so) with blank PROVIDES!",
+ ),
+ noiselevel=-1,
+ level=logging.ERROR,
+ )
if unrecognized_elf_files:
qa_msg = ["QA Notice: Unrecognized ELF file(s):"]
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-10-03 19:31 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-10-03 19:31 UTC (permalink / raw
To: gentoo-commits
commit: f371f1804c5d2d263ca78aa79c3feed1bc5831ae
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Oct 3 17:48:48 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Oct 3 19:30:33 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=f371f180
prepare_build_dirs: handle copytree FileExistsError
Bug: https://bugs.gentoo.org/815871
Reviewed-by: Michał Górny <mgorny <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/prepare_build_dirs.py | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index 41de1a3a9..410c7e4ae 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -477,13 +477,21 @@ def _ensure_log_subdirs(logdir, subdir):
ensure_dirs(current, uid=uid, gid=gid, mode=grp_mode, mask=0)
+def _copytree(src, dst, **kwargs):
+ try:
+ shutil.copytree(src, dst, **kwargs)
+ except FileExistsError:
+ shutil.rmtree(dst)
+ shutil.copytree(src, dst, **kwargs)
+
+
def _prepare_fake_filesdir(settings):
real_filesdir = settings["O"] + "/files"
filesdir = settings["FILESDIR"]
# Copy files from real directory to ebuild directory (without metadata).
if os.path.isdir(real_filesdir):
- shutil.copytree(real_filesdir, filesdir, copy_function=copyfile)
+ _copytree(real_filesdir, filesdir, copy_function=copyfile)
apply_recursive_permissions(
filesdir,
uid=portage_uid,
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-09-28 11:25 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2021-09-28 11:25 UTC (permalink / raw
To: gentoo-commits
commit: 773ba1701f94bdd46086d294efcf97985b67841d
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Tue Sep 28 11:23:16 2021 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Tue Sep 28 11:23:16 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=773ba170
Attempt to fix creating FILESDIR properly
Since we perform a mode fixup on FILESDIR anyway, just let copytree()
create it. This should finally fix all the problems: have the directory
created without errors and work with Python < 3.8.
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/prepare_build_dirs.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index 0ae93a1d5..41de1a3a9 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -480,7 +480,6 @@ def _ensure_log_subdirs(logdir, subdir):
def _prepare_fake_filesdir(settings):
real_filesdir = settings["O"] + "/files"
filesdir = settings["FILESDIR"]
- portage.util.ensure_dirs(filesdir, mode=0o755)
# Copy files from real directory to ebuild directory (without metadata).
if os.path.isdir(real_filesdir):
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-09-28 11:19 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2021-09-28 11:19 UTC (permalink / raw
To: gentoo-commits
commit: cc7d98cec7b410d875d44a5130b22960746aca06
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Tue Sep 28 11:19:20 2021 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Tue Sep 28 11:19:20 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=cc7d98ce
Revert "prepare_build_dirs: Fix copytree() on Python 3.7"
This is (obviously) the wrong fix, resulting in empty FILESDIR.
Reverts: 4c2d9475fe14a8947f4f552a1be0b101bb1eef96
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/prepare_build_dirs.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index 7407122ca..0ae93a1d5 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -483,7 +483,7 @@ def _prepare_fake_filesdir(settings):
portage.util.ensure_dirs(filesdir, mode=0o755)
# Copy files from real directory to ebuild directory (without metadata).
- if os.path.isdir(real_filesdir) and not os.path.isdir(filesdir):
+ if os.path.isdir(real_filesdir):
shutil.copytree(real_filesdir, filesdir, copy_function=copyfile)
apply_recursive_permissions(
filesdir,
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-09-28 7:21 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-09-28 7:21 UTC (permalink / raw
To: gentoo-commits
commit: 6b2f67eddf66eb685cccfad3bec23147f318f420
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Sep 28 00:17:05 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Sep 28 07:19:47 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=6b2f67ed
prepare_build_dirs: apply permissions to filesdir
Bug: https://bugs.gentoo.org/815196
Reviewed-by: Michał Górny <mgorny <AT> gentoo.org>
Reviewed-by: Sam James <sam <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/prepare_build_dirs.py | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index f37c9647b..7407122ca 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -484,8 +484,15 @@ def _prepare_fake_filesdir(settings):
# Copy files from real directory to ebuild directory (without metadata).
if os.path.isdir(real_filesdir) and not os.path.isdir(filesdir):
- shutil.copytree(
- real_filesdir, filesdir, copy_function=copyfile
+ shutil.copytree(real_filesdir, filesdir, copy_function=copyfile)
+ apply_recursive_permissions(
+ filesdir,
+ uid=portage_uid,
+ gid=portage_gid,
+ dirmode=0o750,
+ dirmask=0,
+ filemode=0o640,
+ filemask=0,
)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-09-27 20:51 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2021-09-27 20:51 UTC (permalink / raw
To: gentoo-commits
commit: 4c2d9475fe14a8947f4f552a1be0b101bb1eef96
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Mon Sep 27 20:50:53 2021 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Mon Sep 27 20:50:53 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=4c2d9475
prepare_build_dirs: Fix copytree() on Python 3.7
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/prepare_build_dirs.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index 2e2ef73f4..f37c9647b 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -483,9 +483,9 @@ def _prepare_fake_filesdir(settings):
portage.util.ensure_dirs(filesdir, mode=0o755)
# Copy files from real directory to ebuild directory (without metadata).
- if os.path.isdir(real_filesdir):
+ if os.path.isdir(real_filesdir) and not os.path.isdir(filesdir):
shutil.copytree(
- real_filesdir, filesdir, copy_function=copyfile, dirs_exist_ok=True
+ real_filesdir, filesdir, copy_function=copyfile
)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-09-08 10:17 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2021-09-08 10:17 UTC (permalink / raw
To: gentoo-commits
commit: 505b7d7fc997e71ebe6de17bda24a210f0e65042
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Wed Sep 8 09:32:35 2021 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Wed Sep 8 09:32:35 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=505b7d7f
Extend setuptools warning regexp
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 366cbb9ca..3da239be2 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1870,7 +1870,7 @@ def _check_build_log(mysettings, out=None):
# we deduplicate these since they is repeated for every setup.py call
setuptools_warn = set()
- setuptools_warn_re = re.compile(r'.*\/setuptools\/.*: UserWarning: (.*)')
+ setuptools_warn_re = re.compile(r'.*\/setuptools\/.*: .*Warning: (.*)')
# skip useless version normalization warnings
setuptools_warn_ignore_re = [
re.compile(r'Normalizing .*')
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-08-05 8:47 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2021-08-05 8:47 UTC (permalink / raw
To: gentoo-commits
commit: 95dbf3516c5c1f16212e4d87d42198896e8dade4
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Tue Jul 6 07:47:30 2021 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Thu Aug 5 08:46:53 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=95dbf351
Ignore version normalization warnings from setuptools
The version normalization warnings are not really meaningful and are
quite common when using date-based versioning. Do not report them.
Closes: https://github.com/gentoo/portage/pull/735
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 0cbc2d01b..366cbb9ca 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1871,6 +1871,10 @@ def _check_build_log(mysettings, out=None):
# we deduplicate these since they is repeated for every setup.py call
setuptools_warn = set()
setuptools_warn_re = re.compile(r'.*\/setuptools\/.*: UserWarning: (.*)')
+ # skip useless version normalization warnings
+ setuptools_warn_ignore_re = [
+ re.compile(r'Normalizing .*')
+ ]
def _eerror(lines):
for line in lines:
@@ -1903,7 +1907,12 @@ def _check_build_log(mysettings, out=None):
m = setuptools_warn_re.match(line)
if m is not None:
- setuptools_warn.add(m.group(1))
+ warn_text = m.group(1)
+ for ign in setuptools_warn_ignore_re:
+ if ign.match(warn_text):
+ break
+ else:
+ setuptools_warn.add(warn_text)
except (EOFError, zlib.error) as e:
_eerror(["portage encountered a zlib error: '%s'" % (e,),
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-06-20 18:54 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-06-20 18:54 UTC (permalink / raw
To: gentoo-commits
commit: 2ce11f06e48290efb2d4b6743c8edf01c176b0fc
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun Jun 20 17:53:02 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Jun 20 18:54:21 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=2ce11f06
fetch: Fix AttributeError when no URL specified
Closes: https://github.com/gentoo/portage/pull/732
Bug: https://bugs.gentoo.org/796812
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index c2f7dffc0..100450792 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -907,8 +907,8 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
primaryuri_dict = {}
thirdpartymirror_uris = {}
for myfile, myuri in file_uri_tuples:
- override_mirror = myuri.startswith("mirror+")
- override_fetch = override_mirror or myuri.startswith("fetch+")
+ override_mirror = (myuri or "").startswith("mirror+")
+ override_fetch = override_mirror or (myuri or "").startswith("fetch+")
if override_fetch:
myuri = myuri.partition("+")[2]
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-06-05 18:08 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-06-05 18:08 UTC (permalink / raw
To: gentoo-commits
commit: de877dba8af0d57dcf3767a891daf3fd6908d7ff
Author: Daniel M. Weeks <dan <AT> danweeks <DOT> net>
AuthorDate: Mon May 31 15:46:55 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Jun 5 17:59:56 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=de877dba
Support GLEP 75 layout in GENTOO_MIRRORS
Signed-off-by: Daniel M. Weeks <dan <AT> danweeks.net>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index c2bfca61c..c2f7dffc0 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1253,7 +1253,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
if fsmirrors and not os.path.exists(myfile_path) and has_space:
for mydir in fsmirrors:
- mirror_file = os.path.join(mydir, myfile)
+ mirror_file = get_mirror_url(mydir, myfile, mysettings)
try:
shutil.copyfile(mirror_file, download_path)
writemsg(_("Local mirror has file: %s\n") % myfile)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-06-05 18:08 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-06-05 18:08 UTC (permalink / raw
To: gentoo-commits
commit: e284848704cb5e189eec35a93a54a8e91aa42d64
Author: Daniel M. Weeks <dan <AT> danweeks <DOT> net>
AuthorDate: Mon May 31 15:21:45 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Jun 5 17:59:55 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e2848487
Support GLEP 75 layout in PORTAGE_RO_DISTDIRS
Signed-off-by: Daniel M. Weeks <dan <AT> danweeks.net>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index f687a01a2..c2bfca61c 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -663,8 +663,15 @@ def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
else:
tmpfile = '.layout.conf.%s' % urlparse(mirror_url).hostname
try:
- if fetch({tmpfile: (mirror_url + '/distfiles/layout.conf',)},
- mysettings, force=1, try_mirrors=0):
+ if mirror_url[:1] == "/":
+ tmpfile = os.path.join(mirror_url, "layout.conf")
+ mirror_conf.read_from_file(tmpfile)
+ elif fetch(
+ {tmpfile: (mirror_url + "/distfiles/layout.conf",)},
+ mysettings,
+ force=1,
+ try_mirrors=0,
+ ):
tmpfile = os.path.join(mysettings['DISTDIR'], tmpfile)
mirror_conf.read_from_file(tmpfile)
else:
@@ -683,8 +690,10 @@ def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
path = mirror_conf.get_best_supported_layout(filename=filename).get_path(filename)
if urlparse(mirror_url).scheme in ('ftp', 'http', 'https'):
path = urlquote(path)
- return mirror_url + "/distfiles/" + path
-
+ if mirror_url[:1] == "/":
+ return os.path.join(mirror_url, path)
+ else:
+ return mirror_url + "/distfiles/" + path
def fetch(myuris, mysettings, listonly=0, fetchonly=0,
locks_in_subdir=".locks", use_locks=1, try_mirrors=1, digests=None,
@@ -1212,7 +1221,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
if distdir_writable and ro_distdirs:
readonly_file = None
for x in ro_distdirs:
- filename = os.path.join(x, myfile)
+ filename = get_mirror_url(x, myfile, mysettings)
match, mystat = _check_distfile(
filename, pruned_digests, eout, hash_filter=hash_filter)
if match:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-05-31 19:54 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2021-05-31 19:54 UTC (permalink / raw
To: gentoo-commits
commit: 91e5c15943a8ad01f5133f21730d8800c415a00a
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sat May 29 10:16:59 2021 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Mon May 31 19:53:34 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=91e5c159
Emit QA notices for setuptools warnings
Emit QA notices for setuptools warnings such as:
* QA Notice: setuptools warnings detected:
*
* Usage of dash-separated 'build-requires' will not be supported in future versions. Please use the underscore name 'build_requires' instead
* Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
* Usage of dash-separated 'upload-dir' will not be supported in future versions. Please use the underscore name 'upload_dir' instead
Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Closes: https://github.com/gentoo/portage/pull/722
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 86c1d40b4..b1557edd7 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1863,6 +1863,10 @@ def _check_build_log(mysettings, out=None):
re.compile(r'g?make\[\d+\]: warning: jobserver unavailable:')
make_jobserver = []
+ # we deduplicate these since they is repeated for every setup.py call
+ setuptools_warn = set()
+ setuptools_warn_re = re.compile(r'.*\/setuptools\/.*: UserWarning: (.*)')
+
def _eerror(lines):
for line in lines:
eerror(line, phase="install", key=mysettings.mycpv, out=out)
@@ -1892,6 +1896,10 @@ def _check_build_log(mysettings, out=None):
if make_jobserver_re.match(line) is not None:
make_jobserver.append(line.rstrip("\n"))
+ m = setuptools_warn_re.match(line)
+ if m is not None:
+ setuptools_warn.add(m.group(1))
+
except (EOFError, zlib.error) as e:
_eerror(["portage encountered a zlib error: '%s'" % (e,),
"while reading the log file: '%s'" % logfile])
@@ -1945,6 +1953,12 @@ def _check_build_log(mysettings, out=None):
msg.extend("\t" + line for line in make_jobserver)
_eqawarn(msg)
+ if setuptools_warn:
+ msg = [_("QA Notice: setuptools warnings detected:")]
+ msg.append("")
+ msg.extend("\t" + line for line in sorted(setuptools_warn))
+ _eqawarn(msg)
+
f.close()
if f_real is not None:
f_real.close()
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-05-24 5:25 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-05-24 5:25 UTC (permalink / raw
To: gentoo-commits
commit: af0d91e9a251cb6ecf2568c51b21e6072e8c5241
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Wed May 12 21:37:56 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon May 24 04:56:02 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=af0d91e9
Refactor fetch() for selective restrictions
Refactor the fetch() function so that fetch/mirror restrictions only
affect the code per-URI rather than globally. This will make it easier
to apply restrictions selectively in EAPI 8.
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 54 ++++++++++++++++++-------------------
1 file changed, 26 insertions(+), 28 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 73abec595..2f3111b65 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -813,8 +813,6 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
custommirrors = grabdict(os.path.join(mysettings["PORTAGE_CONFIGROOT"],
CUSTOM_MIRRORS_FILE), recursive=1)
- mymirrors=[]
-
if listonly or ("distlocks" not in features):
use_locks = 0
@@ -832,16 +830,22 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
"FEATURES in /etc/portage/make.conf\n"), noiselevel=-1)
# use_locks = 0
- # local mirrors are always added
- if try_mirrors and "local" in custommirrors:
- mymirrors += custommirrors["local"]
-
- if restrict_mirror:
- # We don't add any mirrors.
- pass
- else:
- if try_mirrors:
- mymirrors += [x.rstrip("/") for x in mysettings["GENTOO_MIRRORS"].split() if x]
+ local_mirrors = []
+ public_mirrors = []
+ fsmirrors = []
+ if try_mirrors:
+ for x in custommirrors.get("local", []):
+ if x.startswith("/"):
+ fsmirrors.append(x)
+ else:
+ local_mirrors.append(x)
+ for x in mysettings["GENTOO_MIRRORS"].split():
+ if not x:
+ continue
+ if x.startswith('/'):
+ fsmirrors.append(x.rstrip("/"))
+ else:
+ public_mirrors.append(x.rstrip("/"))
hash_filter = _hash_filter(mysettings.get("PORTAGE_CHECKSUM_FILTER", ""))
if hash_filter.transparent:
@@ -864,23 +868,8 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
shlex_split(mysettings.get("PORTAGE_RO_DISTDIRS", "")) \
if os.path.isdir(x)]
- fsmirrors = []
- for x in range(len(mymirrors)-1,-1,-1):
- if mymirrors[x] and mymirrors[x][0]=='/':
- fsmirrors += [mymirrors[x]]
- del mymirrors[x]
-
restrict_fetch = "fetch" in restrict
force_mirror = "force-mirror" in features and not restrict_mirror
- custom_local_mirrors = custommirrors.get("local", [])
- if restrict_fetch:
- # With fetch restriction, a normal uri may only be fetched from
- # custom local mirrors (if available). A mirror:// uri may also
- # be fetched from specific mirrors (effectively overriding fetch
- # restriction, but only for specific mirrors).
- locations = custom_local_mirrors
- else:
- locations = mymirrors
file_uri_tuples = []
# Check for 'items' attribute since OrderedDict is not a dict.
@@ -916,7 +905,16 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
".mirror-cache.json")
else:
mirror_cache = None
- for l in locations:
+
+ # With fetch restriction, a normal uri may only be fetched from
+ # custom local mirrors (if available). A mirror:// uri may also
+ # be fetched from specific mirrors (effectively overriding fetch
+ # restriction, but only for specific mirrors).
+ location_lists = [local_mirrors]
+ if not restrict_fetch and not restrict_mirror:
+ location_lists.append(public_mirrors)
+
+ for l in itertools.chain(*location_lists):
filedict[myfile].append(functools.partial(
get_mirror_url, l, myfile, mysettings, mirror_cache))
if myuri is None:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-02-25 9:33 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-02-25 9:33 UTC (permalink / raw
To: gentoo-commits
commit: 48226b2df1408cf1944cb7c6128c73710c740dd3
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Thu Feb 25 09:26:49 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Thu Feb 25 09:31:01 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=48226b2d
fetch.py: fix UnboundLocalError: local variable 'myfile'
Reported-by: Ionen Wolkens <sudinave <AT> gmail.com>
Bug: https://bugs.gentoo.org/772386
Fixes: 3e5ef19d5d6b ("fetch.py: fix UnboundLocalError: local variable 'myfile'")
Fixes: b9ef191c7498 ("MirrorLayoutConfig: content digest support (bug 756778)")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index d23bbfecb..a683793f0 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -898,11 +898,11 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
for myuri in myuris:
if urlparse(myuri).scheme:
file_uri_tuples.append(
- (DistfileName(myfile, digests=mydigests.get(os.path.basename(myuri))), myuri)
+ (DistfileName(os.path.basename(myuri), digests=mydigests.get(os.path.basename(myuri))), myuri)
)
else:
file_uri_tuples.append(
- (DistfileName(myfile, digests=mydigests.get(os.path.basename(myuri))), None)
+ (DistfileName(os.path.basename(myuri), digests=mydigests.get(os.path.basename(myuri))), None)
)
filedict = OrderedDict()
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-02-24 15:14 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-02-24 15:14 UTC (permalink / raw
To: gentoo-commits
commit: 3e5ef19d5d6b26fde08da463d730ff90edf6eb29
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Wed Feb 24 15:08:30 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed Feb 24 15:12:40 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=3e5ef19d
fetch.py: fix UnboundLocalError: local variable 'myfile'
Reported-by: Ionen Wolkens <sudinave <AT> gmail.com>
Bug: https://bugs.gentoo.org/772386
Fixes: b9ef191c7498 ("MirrorLayoutConfig: content digest support (bug 756778)")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index f0ae864ad..d23bbfecb 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -898,11 +898,11 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
for myuri in myuris:
if urlparse(myuri).scheme:
file_uri_tuples.append(
- (DistfileName(myfile, digests=mydigests.get(myfile)), myuri)
+ (DistfileName(myfile, digests=mydigests.get(os.path.basename(myuri))), myuri)
)
else:
file_uri_tuples.append(
- (DistfileName(myfile, digests=mydigests.get(myfile)), None)
+ (DistfileName(myfile, digests=mydigests.get(os.path.basename(myuri))), None)
)
filedict = OrderedDict()
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-02-22 5:32 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-02-22 5:32 UTC (permalink / raw
To: gentoo-commits
commit: 784d2c19f2d62982db16af9055bdab6b595a661b
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Feb 19 05:51:43 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Feb 22 05:26:53 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=784d2c19
make.defaults: prevent USE="${USE} ..." misbehavior
Discard parent profile USE from the expand_map variable before
it is used to evaluate a child profile. This prevents accidents
triggered by USE="${USE} ..." settings at the top of make.defaults
which caused parent profile USE to override parent profile
package.use settings.
Note that would be nice to guard the USE_EXPAND variables like
this too, but unfortunately USE_EXPAND is not known until after
make.defaults has been evaluated, so that will require some form
of make.defaults preprocessing.
Bug: https://bugs.gentoo.org/771549
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index e5ec681af..690efde9d 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -612,9 +612,20 @@ class config:
mygcfg = {}
if profiles_complex:
- mygcfg_dlists = [getconfig(os.path.join(x.location, "make.defaults"),
- tolerant=tolerant, expand=expand_map, recursive=x.portage1_directories)
- for x in profiles_complex]
+ mygcfg_dlists = []
+ for x in profiles_complex:
+ # Prevent accidents triggered by USE="${USE} ..." settings
+ # at the top of make.defaults which caused parent profile
+ # USE to override parent profile package.use settings.
+ # It would be nice to guard USE_EXPAND variables like
+ # this too, but unfortunately USE_EXPAND is not known
+ # until after make.defaults has been evaluated, so that
+ # will require some form of make.defaults preprocessing.
+ expand_map.pop("USE", None)
+ mygcfg_dlists.append(
+ getconfig(os.path.join(x.location, "make.defaults"),
+ tolerant=tolerant, expand=expand_map,
+ recursive=x.portage1_directories))
self._make_defaults = mygcfg_dlists
mygcfg = stack_dicts(mygcfg_dlists,
incrementals=self.incrementals)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-01-17 13:15 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-01-17 13:15 UTC (permalink / raw
To: gentoo-commits
commit: 0aac11200780225cf14a07d6e4445234f1e0f72a
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 17 12:39:15 2021 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Jan 17 13:02:29 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=0aac1120
doebuild: use NamedTemporaryFile for PORTAGE_BINPKG_TMPFILE
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index f6cee4518..476689d5e 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1175,11 +1175,14 @@ def doebuild(myebuild, mydo, _unused=DeprecationWarning, settings=None, debug=0,
# the current user doesn't have write access to $PKGDIR.
if hasattr(portage, 'db'):
bintree = portage.db[mysettings['EROOT']]['bintree']
- mysettings["PORTAGE_BINPKG_TMPFILE"] = \
- bintree.getname(mysettings.mycpv) + \
- ".%s" % (portage.getpid(),)
- bintree._ensure_dir(os.path.dirname(
- mysettings["PORTAGE_BINPKG_TMPFILE"]))
+ binpkg_tmpfile_dir = os.path.join(bintree.pkgdir, mysettings["CATEGORY"])
+ bintree._ensure_dir(binpkg_tmpfile_dir)
+ with tempfile.NamedTemporaryFile(
+ prefix=mysettings["PF"],
+ suffix=".tbz2." + str(portage.getpid()),
+ dir=binpkg_tmpfile_dir,
+ delete=False) as binpkg_tmpfile:
+ mysettings["PORTAGE_BINPKG_TMPFILE"] = binpkg_tmpfile.name
else:
parent_dir = os.path.join(mysettings["PKGDIR"],
mysettings["CATEGORY"])
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2021-01-10 3:24 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2021-01-10 3:24 UTC (permalink / raw
To: gentoo-commits
commit: a362242c8a4e9e44c17c3d284dd2dbfc14e59635
Author: Daniel M. Weeks <dan <AT> danweeks <DOT> net>
AuthorDate: Wed Nov 11 17:27:15 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Jan 10 03:19:10 2021 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=a362242c
Fix varexpand, splitting in fetch
Performing variable expansion after splitting prevents expanding a
variable to an unquoted form:
Given VAR="a b c": "echo \${VAR}" produces ['echo', 'a b c']
It should be possible for a user to control quoting such that:
"echo \"\${VAR}\"" produces ['echo', 'a b c']
"echo \${VAR}" produces ['echo', 'a', 'b', 'c']
Reordering splitting and variable expansion fixes this and mirrors the
order used in _start_gpg_proc of ManifestTask.
Signed-off-by: Daniel M. Weeks <dan <AT> danweeks.net>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index ca031f31e..7be9d0239 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1296,8 +1296,9 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
if v is not None:
variables[k] = v
- myfetch = shlex_split(locfetch)
- myfetch = [varexpand(x, mydict=variables) for x in myfetch]
+ myfetch = varexpand(locfetch, mydict=variables)
+ myfetch = shlex_split(myfetch)
+
myret = -1
try:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-11-02 1:34 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-11-02 1:34 UTC (permalink / raw
To: gentoo-commits
commit: e31c0c929673482101a066541d462d692502a6d1
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Nov 1 20:10:18 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Nov 2 01:30:32 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e31c0c92
make.conf: expand special *ROOT variables (bug 752147)
Bug: https://bugs.gentoo.org/752147
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index a09fdbced..a188c700d 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -401,9 +401,14 @@ class config:
expand_map = env_d.copy()
self._expand_map = expand_map
- # Allow make.globals to set default paths relative to ${EPREFIX}.
+ # Allow make.globals and make.conf to set paths relative to vars like ${EPREFIX}.
+ expand_map["BROOT"] = broot
expand_map["EPREFIX"] = eprefix
+ expand_map["EROOT"] = eroot
+ expand_map["ESYSROOT"] = esysroot
expand_map["PORTAGE_CONFIGROOT"] = config_root
+ expand_map["ROOT"] = target_root
+ expand_map["SYSROOT"] = sysroot
if portage._not_installed:
make_globals_path = os.path.join(PORTAGE_BASE_PATH, "cnf", "make.globals")
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-09-14 7:23 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-09-14 7:23 UTC (permalink / raw
To: gentoo-commits
commit: 20b6a36fbf791bcc7d42bd429cf8116399a014a7
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Sep 12 21:23:28 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Sep 12 22:08:45 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=20b6a36f
get_mirror_url: urlquote only for ftp, http, and https (bug 741474)
It's necessary to use urlquote for correct behavior with ftp, http,
and https, since it's possible for file names to contain percent
encoded characters that need to be protected by an additional layer
of percent encoding. For other protocols such as sftp and rsync,
all characters are interpreted literally, so urlquote must not be
used.
Fixes: c238d5f7ed264179c263f5a2da983c4ee50b4f00
Bug: https://bugs.gentoo.org/719810
Bug: https://bugs.gentoo.org/741474
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 7c61fe463..ca031f31e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -513,8 +513,12 @@ def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
json.dump(cache, f)
f.close()
- return (mirror_url + "/distfiles/" +
- urlquote(mirror_conf.get_best_supported_layout().get_path(filename)))
+ # For some protocols, urlquote is required for correct behavior,
+ # and it must not be used for other protocols like rsync and sftp.
+ path = mirror_conf.get_best_supported_layout().get_path(filename)
+ if urlparse(mirror_url).scheme in ('ftp', 'http', 'https'):
+ path = urlquote(path)
+ return mirror_url + "/distfiles/" + path
def fetch(myuris, mysettings, listonly=0, fetchonly=0,
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-08-03 21:42 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-08-03 21:42 UTC (permalink / raw
To: gentoo-commits
commit: 65a642eea83b50b8cf16a1cae5088db0a68383f2
Author: Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug 3 20:20:28 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug 3 21:25:53 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=65a642ee
lib/portage/package/ebuild/getmaskingreason.py: drop unused-import
* Drop unused-import
* Update copyright
Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/getmaskingreason.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/getmaskingreason.py b/lib/portage/package/ebuild/getmaskingreason.py
index 22232de29..9cda20456 100644
--- a/lib/portage/package/ebuild/getmaskingreason.py
+++ b/lib/portage/package/ebuild/getmaskingreason.py
@@ -1,4 +1,4 @@
-# Copyright 2010-2014 Gentoo Foundation
+# Copyright 2010-2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
__all__ = ['getmaskingreason']
@@ -10,7 +10,7 @@ from portage.dep import Atom, match_from_list
from portage.exception import InvalidAtom
from portage.localization import _
from portage.repository.config import _gen_valid_repo
-from portage.util import grablines, normalize_path
+from portage.util import grablines
from portage.versions import catpkgsplit, _pkg_str
def getmaskingreason(mycpv, metadata=None, settings=None,
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-08-03 21:42 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-08-03 21:42 UTC (permalink / raw
To: gentoo-commits
commit: 06cd15d01685ab3f944ba0b9edade6ba67cdc2da
Author: Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug 3 20:20:27 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug 3 21:25:53 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=06cd15d0
lib/portage/package/ebuild/prepare_build_dirs.py: drop unused-import
Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/prepare_build_dirs.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index b63cd89fa..080d5309f 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -18,7 +18,6 @@ from portage.output import colorize
from portage.util import apply_recursive_permissions, \
apply_secpass_permissions, ensure_dirs, normalize_path, writemsg
from portage.util.install_mask import _raise_exc
-from portage.const import EPREFIX
def prepare_build_dirs(myroot=None, settings=None, cleanup=False):
"""
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-08-03 19:30 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-08-03 19:30 UTC (permalink / raw
To: gentoo-commits
commit: ccb08d292503389635cf9183f79218ca02c12b01
Author: Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug 3 19:05:45 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug 3 19:21:31 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=ccb08d29
lib/portage/package/ebuild/config.py: drop unused-import
* Fix unused-import
* Update copyright
Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 7c5f82e0c..74b998d0b 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -34,13 +34,11 @@ from portage.const import CACHE_PATH, \
PRIVATE_PATH, PROFILE_PATH, USER_CONFIG_PATH, \
USER_VIRTUALS_FILE
from portage.dbapi import dbapi
-from portage.dbapi.porttree import portdbapi
from portage.dep import Atom, isvalidatom, match_from_list, use_reduce, _repo_separator, _slot_separator
from portage.eapi import (eapi_exports_AA, eapi_exports_merge_type,
eapi_supports_prefix, eapi_exports_replace_vars, _get_eapi_attrs)
from portage.env.loaders import KeyValuePairFileLoader
-from portage.exception import InvalidDependString, IsADirectory, \
- PortageException
+from portage.exception import InvalidDependString, PortageException
from portage.localization import _
from portage.output import colorize
from portage.process import fakeroot_capable, sandbox_capable
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-08-03 19:30 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-08-03 19:30 UTC (permalink / raw
To: gentoo-commits
commit: 89b9dd50bc6b6cf0bb30c8caef2383734805caae
Author: Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug 3 19:05:50 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug 3 19:21:57 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=89b9dd50
lib/portage/package/ebuild/getmaskingstatus.py: drop unused-import
Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/getmaskingstatus.py | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/lib/portage/package/ebuild/getmaskingstatus.py b/lib/portage/package/ebuild/getmaskingstatus.py
index cf2b7344b..c008a0d03 100644
--- a/lib/portage/package/ebuild/getmaskingstatus.py
+++ b/lib/portage/package/ebuild/getmaskingstatus.py
@@ -3,14 +3,12 @@
__all__ = ['getmaskingstatus']
-
import portage
from portage import eapi_is_supported, _eapi_is_deprecated
from portage.exception import InvalidDependString
from portage.localization import _
from portage.package.ebuild.config import config
-from portage.versions import catpkgsplit, _pkg_str
-
+from portage.versions import _pkg_str
class _UnmaskHint:
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-08-03 19:30 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-08-03 19:30 UTC (permalink / raw
To: gentoo-commits
commit: 5e7440acab6831b1e8c06c2f46794ec8b13b94ab
Author: Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug 3 19:05:51 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug 3 19:21:54 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=5e7440ac
lib/portage/package/ebuild/_spawn_nofetch.py: drop unused-import
* Drop unused-import
* Update copyright
Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/_spawn_nofetch.py | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/lib/portage/package/ebuild/_spawn_nofetch.py b/lib/portage/package/ebuild/_spawn_nofetch.py
index 980d7f629..c7cf8d29b 100644
--- a/lib/portage/package/ebuild/_spawn_nofetch.py
+++ b/lib/portage/package/ebuild/_spawn_nofetch.py
@@ -1,9 +1,8 @@
-# Copyright 2010-2019 Gentoo Authors
+# Copyright 2010-2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
import tempfile
-import portage
from portage import os
from portage import shutil
from portage.const import EBUILD_PHASES
@@ -13,12 +12,9 @@ from portage.package.ebuild.doebuild import doebuild_environment
from portage.package.ebuild.prepare_build_dirs import prepare_build_dirs
from portage.util.futures import asyncio
from portage.util._async.SchedulerInterface import SchedulerInterface
-from portage.util._eventloop.EventLoop import EventLoop
-from portage.util._eventloop.global_event_loop import global_event_loop
from _emerge.CompositeTask import CompositeTask
from _emerge.EbuildPhase import EbuildPhase
-
class SpawnNofetchWithoutBuilddir(CompositeTask):
"""
This spawns pkg_nofetch if appropriate, while avoiding the
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-05-31 23:58 Mike Gilbert
0 siblings, 0 replies; 93+ messages in thread
From: Mike Gilbert @ 2020-05-31 23:58 UTC (permalink / raw
To: gentoo-commits
commit: c238d5f7ed264179c263f5a2da983c4ee50b4f00
Author: Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Sun May 31 21:24:25 2020 +0000
Commit: Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Sun May 31 21:30:14 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=c238d5f7
Escape percent-signs in portage.package.ebuild.fetch.get_mirror_url()
This avoids double-escaping in emirrordist. We only want to escape the
path when fetching the file from the mirror, not when mirroring the
file.
Bug: https://bugs.gentoo.org/719810
Fixes: 4c18f523bb86a8be4c148f365dabee06fca2e4fa
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 28e7caf53..9682fea89 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -26,6 +26,11 @@ try:
except ImportError:
from urlparse import urlparse
+try:
+ from urllib.parse import quote as urlquote
+except ImportError:
+ from urllib import quote as urlquote
+
import portage
portage.proxy.lazyimport.lazyimport(globals(),
'portage.package.ebuild.config:check_config_instance,config',
@@ -520,7 +525,7 @@ def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
f.close()
return (mirror_url + "/distfiles/" +
- mirror_conf.get_best_supported_layout().get_path(filename))
+ urlquote(mirror_conf.get_best_supported_layout().get_path(filename)))
def fetch(myuris, mysettings, listonly=0, fetchonly=0,
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-05-31 21:17 Mike Gilbert
0 siblings, 0 replies; 93+ messages in thread
From: Mike Gilbert @ 2020-05-31 21:17 UTC (permalink / raw
To: gentoo-commits
commit: 54d572cbc3ef2b646283b826813a47e5268bf734
Author: Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Sun May 31 21:15:29 2020 +0000
Commit: Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Sun May 31 21:15:29 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=54d572cb
Revert "Escape percent-signs in filename when fetching from mirrors"
This probably breaks emirrordist; we will need separate methods for
layout.get_path() when it is used client-side.
This reverts commit 4c18f523bb86a8be4c148f365dabee06fca2e4fa.
Bug: https://bugs.gentoo.org/719810
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 47c3ad28f..28e7caf53 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -26,11 +26,6 @@ try:
except ImportError:
from urlparse import urlparse
-try:
- from urllib.parse import quote as urlquote
-except ImportError:
- from urllib import quote as urlquote
-
import portage
portage.proxy.lazyimport.lazyimport(globals(),
'portage.package.ebuild.config:check_config_instance,config',
@@ -356,7 +351,7 @@ _size_suffix_map = {
class FlatLayout(object):
def get_path(self, filename):
- return urlquote(filename)
+ return filename
def get_filenames(self, distdir):
for dirpath, dirnames, filenames in os.walk(distdir,
@@ -387,7 +382,7 @@ class FilenameHashLayout(object):
c = c // 4
ret += fnhash[:c] + '/'
fnhash = fnhash[c:]
- return ret + urlquote(filename)
+ return ret + filename
def get_filenames(self, distdir):
pattern = ''
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-05-31 20:34 Mike Gilbert
0 siblings, 0 replies; 93+ messages in thread
From: Mike Gilbert @ 2020-05-31 20:34 UTC (permalink / raw
To: gentoo-commits
commit: 4c18f523bb86a8be4c148f365dabee06fca2e4fa
Author: Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Sun May 31 13:11:41 2020 +0000
Commit: Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Sun May 31 20:32:46 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=4c18f523
Escape percent-signs in filename when fetching from mirrors
Bug: https://bugs.gentoo.org/719810
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 28e7caf53..47c3ad28f 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -26,6 +26,11 @@ try:
except ImportError:
from urlparse import urlparse
+try:
+ from urllib.parse import quote as urlquote
+except ImportError:
+ from urllib import quote as urlquote
+
import portage
portage.proxy.lazyimport.lazyimport(globals(),
'portage.package.ebuild.config:check_config_instance,config',
@@ -351,7 +356,7 @@ _size_suffix_map = {
class FlatLayout(object):
def get_path(self, filename):
- return filename
+ return urlquote(filename)
def get_filenames(self, distdir):
for dirpath, dirnames, filenames in os.walk(distdir,
@@ -382,7 +387,7 @@ class FilenameHashLayout(object):
c = c // 4
ret += fnhash[:c] + '/'
fnhash = fnhash[c:]
- return ret + filename
+ return ret + urlquote(filename)
def get_filenames(self, distdir):
pattern = ''
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-05-18 19:23 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2020-05-18 19:23 UTC (permalink / raw
To: gentoo-commits
commit: 878eddfea083eda5aea60510aa1c4e9145f26b99
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 19:22:17 2020 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Mon May 18 19:22:17 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=878eddfe
Revert "doebuild: Use ccache/distcc/icecream only in src_* phases"
This breaks ccache/etc. entirely as they are no longer present in PATH
in later phases.
Reverts: 61f76236aa10c297f79c2fee123a79d26bfa9327
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 9 +--------
1 file changed, 1 insertion(+), 8 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 1343eca6d..2bff94cb1 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -486,14 +486,7 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
distcc = "distcc" in mysettings.features
icecream = "icecream" in mysettings.features
- # run cc thingies only in src_* phases as otherwise they might
- # create files with incorrect ownership and trip over
- # (NB: install has a similar issue but 1) some badly written
- # packages rebuild stuff there, so ccache is very helpful
- # and 2) almost always some compilation will happen earlier,
- # so permissions will be already set correctly)
- if (ccache or distcc or icecream) and mydo in ('unpack',
- 'prepare', 'configure', 'test', 'install'):
+ if ccache or distcc or icecream:
libdir = None
default_abi = mysettings.get("DEFAULT_ABI")
if default_abi:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-05-06 6:37 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2020-05-06 6:37 UTC (permalink / raw
To: gentoo-commits
commit: 61f76236aa10c297f79c2fee123a79d26bfa9327
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Wed May 6 04:54:03 2020 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Wed May 6 06:37:12 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=61f76236
doebuild: Use ccache/distcc/icecream only in src_* phases
Enable ccache/distcc/icecream only when src_* phases are executed.
There is generally little value from them in pkg_*, as these phases
use CC only to do a few tests at most. These compilations generally
are not run in parallel and the programs are tiny, so usually
the overhead exceeds the gain.
On the other hand, running them in pkg_* phases results in files being
created with root ownership that breaks them afterwards when using
userpriv. As a result, user not only loses the benefit of, say, distcc
but also frequently hits error messages that confuse some configure
scripts and result in miscompiled packages.
Bug: https://bugs.gentoo.org/581880
Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Closes: https://github.com/gentoo/portage/pull/547
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 2bff94cb1..1343eca6d 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -486,7 +486,14 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
distcc = "distcc" in mysettings.features
icecream = "icecream" in mysettings.features
- if ccache or distcc or icecream:
+ # run cc thingies only in src_* phases as otherwise they might
+ # create files with incorrect ownership and trip over
+ # (NB: install has a similar issue but 1) some badly written
+ # packages rebuild stuff there, so ccache is very helpful
+ # and 2) almost always some compilation will happen earlier,
+ # so permissions will be already set correctly)
+ if (ccache or distcc or icecream) and mydo in ('unpack',
+ 'prepare', 'configure', 'test', 'install'):
libdir = None
default_abi = mysettings.get("DEFAULT_ABI")
if default_abi:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-03-23 1:49 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-03-23 1:49 UTC (permalink / raw
To: gentoo-commits
commit: 90b7d3b00d5a5eb9b732fb8b5bf0870eb89ab422
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Mar 22 21:42:42 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Mar 23 01:16:09 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=90b7d3b0
_hide_url_passwd: replace all occurrences (bug 713726)
Adjust the regular expression to avoid overly-greedy .+ groups,
so that is will properly replace all occurrences, as necessary
for the purposes of bug 713726 since PORTAGE_BINHOST may contain
multiple values.
Bug: https://bugs.gentoo.org/713726
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index f7984130f..28e7caf53 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -67,7 +67,7 @@ _userpriv_spawn_kwargs = (
)
def _hide_url_passwd(url):
- return re.sub(r'//(.+):.+@(.+)', r'//\1:*password*@\2', url)
+ return re.sub(r'//([^:\s]+):[^@\s]+@', r'//\1:*password*@', url)
def _want_userfetch(settings):
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-03-23 1:40 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-03-23 1:40 UTC (permalink / raw
To: gentoo-commits
commit: 2342e77eea8e00a572623f86d20cde2dc7098aa5
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Mar 22 01:13:22 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Mar 23 01:15:04 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=2342e77e
_post_phase_userpriv_perms: handle $HOME (bug 713100)
Ensure that the userpriv UID has appropriate permission for files
created in $HOME during privileged phases like pkg_setup, in the
same way as for $T. This prevents potential permission issues for
programs invoked during unprivileged phases, and it improves
alignment with PMS which specifies identical behavior for both
$HOME and $T.
Bug: https://bugs.gentoo.org/713100
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 75fcb8a51..2bff94cb1 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1765,9 +1765,10 @@ def _post_phase_userpriv_perms(mysettings):
if "userpriv" in mysettings.features and secpass >= 2:
""" Privileged phases may have left files that need to be made
writable to a less privileged user."""
- apply_recursive_permissions(mysettings["T"],
- uid=portage_uid, gid=portage_gid, dirmode=0o700, dirmask=0,
- filemode=0o600, filemask=0)
+ for path in (mysettings["HOME"], mysettings["T"]):
+ apply_recursive_permissions(path,
+ uid=portage_uid, gid=portage_gid, dirmode=0o700, dirmask=0,
+ filemode=0o600, filemask=0)
def _check_build_log(mysettings, out=None):
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-03-15 1:24 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-03-15 1:24 UTC (permalink / raw
To: gentoo-commits
commit: ef8c21e59953aa5fdd153f19b7bf04356e2164fe
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Mar 15 00:01:25 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Mar 15 00:42:43 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=ef8c21e5
_prepare_workdir: apply PORTAGE_WORKDIR_MODE to PORTAGE_BUILDDIR (bug 692492)
Apply secure PORTAGE_WORKDIR_MODE permissions to PORTAGE_BUILDDIR,
since the child directory ${D} and its children may have vulnerable
permissions as reported in bug 692492.
Bug: https://bugs.gentoo.org/692492
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/prepare_build_dirs.py | 21 ++++++++++++++-------
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/lib/portage/package/ebuild/prepare_build_dirs.py b/lib/portage/package/ebuild/prepare_build_dirs.py
index c325819d1..8349d306f 100644
--- a/lib/portage/package/ebuild/prepare_build_dirs.py
+++ b/lib/portage/package/ebuild/prepare_build_dirs.py
@@ -1,4 +1,4 @@
-# Copyright 2010-2018 Gentoo Foundation
+# Copyright 2010-2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
from __future__ import unicode_literals
@@ -84,7 +84,7 @@ def prepare_build_dirs(myroot=None, settings=None, cleanup=False):
except PortageException:
if not os.path.isdir(mydir):
raise
- for dir_key in ("PORTAGE_BUILDDIR", "HOME", "PKG_LOGDIR", "T"):
+ for dir_key in ("HOME", "PKG_LOGDIR", "T"):
ensure_dirs(mysettings[dir_key], mode=0o755)
apply_secpass_permissions(mysettings[dir_key],
uid=portage_uid, gid=portage_gid)
@@ -272,11 +272,18 @@ def _prepare_workdir(mysettings):
writemsg(_("!!! Unable to parse PORTAGE_WORKDIR_MODE='%s', using %s.\n") % \
(mysettings["PORTAGE_WORKDIR_MODE"], oct(workdir_mode)))
mysettings["PORTAGE_WORKDIR_MODE"] = oct(workdir_mode).replace('o', '')
- try:
- apply_secpass_permissions(mysettings["WORKDIR"],
- uid=portage_uid, gid=portage_gid, mode=workdir_mode)
- except FileNotFound:
- pass # ebuild.sh will create it
+
+ permissions = {'mode': workdir_mode}
+ if portage.data.secpass >= 2:
+ permissions['uid'] = portage_uid
+ if portage.data.secpass >= 1:
+ permissions['gid'] = portage_gid
+
+ # Apply PORTAGE_WORKDIR_MODE to PORTAGE_BUILDDIR, since the child
+ # directory ${D} and its children may have vulnerable permissions
+ # as reported in bug 692492.
+ ensure_dirs(mysettings["PORTAGE_BUILDDIR"], **permissions)
+ ensure_dirs(mysettings["WORKDIR"], **permissions)
if mysettings.get("PORTAGE_LOGDIR", "") == "":
while "PORTAGE_LOGDIR" in mysettings:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-03-02 16:53 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-03-02 16:53 UTC (permalink / raw
To: gentoo-commits
commit: 0bf72716cacc7f79abb61ad3ce33332c743809c9
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Mar 2 16:49:33 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Mar 2 16:51:15 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=0bf72716
doebuild: avoid emerge --pretend --fetchonly event loop recursion
Fixes: ec654122c0eb ("fetch: drop privileges early for NFS root_squash (bug 601252)")
Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 384a3e632..75fcb8a51 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1095,8 +1095,16 @@ def doebuild(myebuild, mydo, _unused=DeprecationWarning, settings=None, debug=0,
digests=dist_digests)
loop = asyncio._safe_loop()
- if not loop.run_until_complete(loop.run_in_executor(ForkExecutor(loop=loop),
- _fetch_subprocess, fetchme, mysettings, listonly, dist_digests)):
+ if loop.is_running():
+ # Called by EbuildFetchonly for emerge --pretend --fetchonly.
+ success = fetch(fetchme, mysettings, listonly=listonly,
+ fetchonly=fetchonly, allow_missing_digests=False,
+ digests=dist_digests)
+ else:
+ success = loop.run_until_complete(
+ loop.run_in_executor(ForkExecutor(loop=loop),
+ _fetch_subprocess, fetchme, mysettings, listonly, dist_digests))
+ if not success:
# Since listonly mode is called by emerge --pretend in an
# asynchronous context, spawn_nofetch would trigger event loop
# recursion here, therefore delegate execution of pkg_nofetch
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-03-02 4:48 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-03-02 4:48 UTC (permalink / raw
To: gentoo-commits
commit: 7adc9b05fc96382d583828f660c9c2e2c2323f22
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Mar 2 04:42:12 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Mar 2 04:43:57 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=7adc9b05
_check_build_log: convert zlib EOFError to eerror message (bug 711174)
Display a non-fatal eerror message when EOFError is encountered,
since this need not be fatal.
Bug: https://bugs.gentoo.org/711174
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 71e3a74ce..384a3e632 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1884,7 +1884,7 @@ def _check_build_log(mysettings, out=None):
if make_jobserver_re.match(line) is not None:
make_jobserver.append(line.rstrip("\n"))
- except zlib.error as e:
+ except (EOFError, zlib.error) as e:
_eerror(["portage encountered a zlib error: '%s'" % (e,),
"while reading the log file: '%s'" % logfile])
finally:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-02-15 20:44 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-02-15 20:44 UTC (permalink / raw
To: gentoo-commits
commit: d8d02bd82f11921db5140941cef4077359b0211d
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 15 20:37:39 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Feb 15 20:42:24 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=d8d02bd8
fetch: remove secpass import (bug 601252)
Refer to the portage.secpass.data attribute directly rather than via
import value, since the value will change after fork and os.setuid
calls when the fetch function is called with dropped privileges as
discussed in bug 601252.
Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index e0260829b..06118b1a6 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1,4 +1,4 @@
-# Copyright 2010-2019 Gentoo Authors
+# Copyright 2010-2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
from __future__ import print_function
@@ -46,7 +46,7 @@ from portage.checksum import (get_valid_checksum_keys, perform_md5, verify_all,
checksum_str)
from portage.const import BASH_BINARY, CUSTOM_MIRRORS_FILE, \
GLOBAL_CONFIG_PATH
-from portage.data import portage_gid, portage_uid, secpass, userpriv_groups
+from portage.data import portage_gid, portage_uid, userpriv_groups
from portage.exception import FileNotFound, OperationNotPermitted, \
PortageException, TryAgain
from portage.localization import _
@@ -882,7 +882,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
if not has_space_superuser:
has_space = False
- elif secpass < 2:
+ elif portage.data.secpass < 2:
has_space = False
elif userfetch:
has_space = False
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2020-02-15 20:28 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2020-02-15 20:28 UTC (permalink / raw
To: gentoo-commits
commit: 9b07545003fed649b1a0a8a9e5cb69d5e2fa4951
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 15 20:23:23 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Feb 15 20:25:34 2020 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=9b075450
fetch: split out _ensure_distdir function (bug 601252)
Split out an _ensure_distdir function which will have to be
called earlier if the fetch function is called with dropped
privileges as discussed in bug 601252.
Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 102 +++++++++++++++++++-----------------
1 file changed, 55 insertions(+), 47 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 7ab054874..e0260829b 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -152,6 +152,59 @@ def _userpriv_test_write_file(settings, file_path):
_userpriv_test_write_file_cache[file_path] = rval
return rval
+
+def _ensure_distdir(settings, distdir):
+ """
+ Ensure that DISTDIR exists with appropriate permissions.
+
+ @param settings: portage config
+ @type settings: portage.package.ebuild.config.config
+ @param distdir: DISTDIR path
+ @type distdir: str
+ @raise PortageException: portage.exception wrapper exception
+ """
+ global _userpriv_test_write_file_cache
+ dirmode = 0o070
+ filemode = 0o60
+ modemask = 0o2
+ dir_gid = portage_gid
+ if "FAKED_MODE" in settings:
+ # When inside fakeroot, directories with portage's gid appear
+ # to have root's gid. Therefore, use root's gid instead of
+ # portage's gid to avoid spurrious permissions adjustments
+ # when inside fakeroot.
+ dir_gid = 0
+
+ userfetch = portage.data.secpass >= 2 and "userfetch" in settings.features
+ userpriv = portage.data.secpass >= 2 and "userpriv" in settings.features
+ write_test_file = os.path.join(distdir, ".__portage_test_write__")
+
+ try:
+ st = os.stat(distdir)
+ except OSError:
+ st = None
+
+ if st is not None and stat.S_ISDIR(st.st_mode):
+ if not (userfetch or userpriv):
+ return
+ if _userpriv_test_write_file(settings, write_test_file):
+ return
+
+ _userpriv_test_write_file_cache.pop(write_test_file, None)
+ if ensure_dirs(distdir, gid=dir_gid, mode=dirmode, mask=modemask):
+ if st is None:
+ # The directory has just been created
+ # and therefore it must be empty.
+ return
+ writemsg(_("Adjusting permissions recursively: '%s'\n") % distdir,
+ noiselevel=-1)
+ if not apply_recursive_permissions(distdir,
+ gid=dir_gid, dirmode=dirmode, dirmask=modemask,
+ filemode=filemode, filemask=modemask, onerror=_raise_exc):
+ raise OperationNotPermitted(
+ _("Failed to apply recursive permissions for the portage group."))
+
+
def _checksum_failure_temp_file(settings, distdir, basename):
"""
First try to find a duplicate temp file with the same checksum and return
@@ -486,9 +539,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
features = mysettings.features
restrict = mysettings.get("PORTAGE_RESTRICT","").split()
-
- userfetch = secpass >= 2 and "userfetch" in features
- userpriv = secpass >= 2 and "userpriv" in features
+ userfetch = portage.data.secpass >= 2 and "userfetch" in features
# 'nomirror' is bad/negative logic. You Restrict mirroring, not no-mirroring.
restrict_mirror = "mirror" in restrict or "nomirror" in restrict
@@ -728,51 +779,8 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
can_fetch = False
if can_fetch and not fetch_to_ro:
- global _userpriv_test_write_file_cache
- dirmode = 0o070
- filemode = 0o60
- modemask = 0o2
- dir_gid = portage_gid
- if "FAKED_MODE" in mysettings:
- # When inside fakeroot, directories with portage's gid appear
- # to have root's gid. Therefore, use root's gid instead of
- # portage's gid to avoid spurrious permissions adjustments
- # when inside fakeroot.
- dir_gid = 0
- distdir_dirs = [""]
try:
-
- for x in distdir_dirs:
- mydir = os.path.join(mysettings["DISTDIR"], x)
- write_test_file = os.path.join(
- mydir, ".__portage_test_write__")
-
- try:
- st = os.stat(mydir)
- except OSError:
- st = None
-
- if st is not None and stat.S_ISDIR(st.st_mode):
- if not (userfetch or userpriv):
- continue
- if _userpriv_test_write_file(mysettings, write_test_file):
- continue
-
- _userpriv_test_write_file_cache.pop(write_test_file, None)
- if ensure_dirs(mydir, gid=dir_gid, mode=dirmode, mask=modemask):
- if st is None:
- # The directory has just been created
- # and therefore it must be empty.
- continue
- writemsg(_("Adjusting permissions recursively: '%s'\n") % mydir,
- noiselevel=-1)
- def onerror(e):
- raise # bail out on the first error that occurs during recursion
- if not apply_recursive_permissions(mydir,
- gid=dir_gid, dirmode=dirmode, dirmask=modemask,
- filemode=filemode, filemask=modemask, onerror=onerror):
- raise OperationNotPermitted(
- _("Failed to apply recursive permissions for the portage group."))
+ _ensure_distdir(mysettings, mysettings["DISTDIR"])
except PortageException as e:
if not os.path.isdir(mysettings["DISTDIR"]):
writemsg("!!! %s\n" % str(e), noiselevel=-1)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-12-09 7:03 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-12-09 7:03 UTC (permalink / raw
To: gentoo-commits
commit: 2022595f042458e1561e0c3ee8f9f987eb5f27b0
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Dec 9 06:56:07 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Dec 9 07:00:30 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=2022595f
fetch: skip _checksum_failure_temp_file in force mode
Avoid files like this in DISTDIR:
$ md5sum .layout*
92dbdc707e5a40c7bfcbd1883173076a .layout.conf.distfiles.gentoo.org
92dbdc707e5a40c7bfcbd1883173076a .layout.conf.distfiles.gentoo.org._checksum_failure_.yrje0wk5
Reported-by: Arfrever Frehtes Taifersar Arahesis <Arfrever <AT> Apache.Org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 4612532c5..7ab054874 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -935,7 +935,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
level=logging.ERROR, noiselevel=-1)
return 0
- if distdir_writable:
+ if distdir_writable and not force:
# Since _check_distfile did not match above, the file
# is either corrupt or its identity has changed since
# the last time it was fetched, so rename it.
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-10-29 1:17 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-10-29 1:17 UTC (permalink / raw
To: gentoo-commits
commit: 1ca5b822133171b131cef3dc15dc43583893ad6b
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Oct 29 00:56:47 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Oct 29 01:07:45 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=1ca5b822
fetch: remove symlink created by FETCHCOMMAND_RSYNC (bug 698046)
This avoids confusing "No such file or directory" errors as demonstrated
by the following test case:
$ ln -s /foo/bar /tmp/sudo-1.8.29rc1.tar.gz
$ wget http://distfiles.gentoo.org/distfiles/sudo-1.8.29rc1.tar.gz -O /tmp/sudo-1.8.29rc1.tar.gz
/tmp/sudo-1.8.29rc1.tar.gz: No such file or directory
Bug: https://bugs.gentoo.org/698046
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 58718b568..4612532c5 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -1283,7 +1283,8 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
# trust the return value from the fetcher. Remove the
# empty file and try to download again.
try:
- if os.stat(download_path).st_size == 0:
+ mystat = os.lstat(download_path)
+ if mystat.st_size == 0 or (stat.S_ISLNK(mystat.st_mode) and not os.path.exists(download_path)):
os.unlink(download_path)
fetched = 0
continue
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-10-24 19:31 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-10-24 19:31 UTC (permalink / raw
To: gentoo-commits
commit: 26fd7ffdd5b74af3aeedf0e6a87ac6b3d1243848
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Thu Oct 24 19:28:40 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Thu Oct 24 19:29:26 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=26fd7ffd
fetch: respect try_mirrors parameter for local mirrors
This is needed in order to prevent infinit recursion like this:
File "portage/package/ebuild/fetch.py", line 1131, in fetch
loc = loc()
File "portage/package/ebuild/fetch.py", line 424, in get_mirror_url
mysettings, force=1, try_mirrors=0):
File "portage/package/ebuild/fetch.py", line 1131, in fetch
loc = loc()
Bug: https://bugs.gentoo.org/698474
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index cf073436e..58718b568 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -588,7 +588,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
# use_locks = 0
# local mirrors are always added
- if "local" in custommirrors:
+ if try_mirrors and "local" in custommirrors:
mymirrors += custommirrors["local"]
if restrict_mirror:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-10-21 17:49 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-10-21 17:49 UTC (permalink / raw
To: gentoo-commits
commit: db78fff2ffc15c5e245d87c2944c869dbe260a78
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 21 17:48:11 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 21 17:48:53 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=db78fff2
FilenameHashLayout.get_filenames: force UTF8 encoding for iglob argument
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index ac25e4326..cf073436e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -307,7 +307,7 @@ class FilenameHashLayout(object):
c = c // 4
pattern += c * '[0-9a-f]' + '/'
pattern += '*'
- for x in glob.iglob(os.path.join(distdir, pattern)):
+ for x in glob.iglob(portage._unicode_encode(os.path.join(distdir, pattern), errors='strict')):
try:
yield portage._unicode_decode(x, errors='strict').rsplit('/', 1)[1]
except UnicodeDecodeError:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-10-20 9:26 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2019-10-20 9:26 UTC (permalink / raw
To: gentoo-commits
commit: 9115e1f6ba35cdcd85f2292dc293f0696caa8f12
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Fri Oct 18 07:23:02 2019 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Sun Oct 20 09:25:10 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=9115e1f6
fetch: Use distfile fetching method to get layout.conf
Rewrite the layout.conf getter to reuse the standard fetch() method
rather than using urlopen(). While at it, fix negative cache elision
to apply to memory cache as well (and not get written to disk if next
mirror was fine).
Most importantly, this ensures that we respect FETCHCOMMAND while
fetching layout.conf, and so layout.conf is fetched the same way normal
distfiles are. With some uncommon configurations, the previous disjoint
logic might have resulted in one of the fetches failing while the other
succeeded.
This also adds some nice verbosity. If mirror connection takes a while,
the user sees that rather than having Portage wait silently.
Bug: https://bugs.gentoo.org/697566
Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 33 ++++++++++++++++-----------------
1 file changed, 16 insertions(+), 17 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 05de12740..cedf12b19 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -382,7 +382,7 @@ class MirrorLayoutConfig(object):
return ret
-def get_mirror_url(mirror_url, filename, cache_path=None):
+def get_mirror_url(mirror_url, filename, mysettings, cache_path=None):
"""
Get correct fetch URL for a given file, accounting for mirror
layout configuration.
@@ -408,23 +408,22 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
if ts >= time.time() - 86400:
mirror_conf.deserialize(data)
else:
+ tmpfile = '.layout.conf.%s' % urlparse(mirror_url).hostname
try:
- f = urlopen(mirror_url + '/distfiles/layout.conf')
- try:
- data = io.StringIO(f.read().decode('utf8'))
- finally:
- f.close()
-
- mirror_conf.read_from_file(data)
+ if fetch({tmpfile: (mirror_url + '/distfiles/layout.conf',)},
+ mysettings, force=1, try_mirrors=0):
+ tmpfile = os.path.join(mysettings['DISTDIR'], tmpfile)
+ mirror_conf.read_from_file(tmpfile)
+ else:
+ raise IOError()
except (ConfigParserError, IOError, UnicodeDecodeError):
- # Do not cache negative results.
- cache_path = None
-
- cache[mirror_url] = (time.time(), mirror_conf.serialize())
- if cache_path is not None:
- f = atomic_ofstream(cache_path, 'w')
- json.dump(cache, f)
- f.close()
+ pass
+ else:
+ cache[mirror_url] = (time.time(), mirror_conf.serialize())
+ if cache_path is not None:
+ f = atomic_ofstream(cache_path, 'w')
+ json.dump(cache, f)
+ f.close()
return (mirror_url + "/distfiles/" +
mirror_conf.get_best_supported_layout().get_path(filename))
@@ -656,7 +655,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
mirror_cache = None
for l in locations:
filedict[myfile].append(functools.partial(
- get_mirror_url, l, myfile, mirror_cache))
+ get_mirror_url, l, myfile, mysettings, mirror_cache))
if myuri is None:
continue
if myuri[:9]=="mirror://":
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-10-19 23:52 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-10-19 23:52 UTC (permalink / raw
To: gentoo-commits
commit: d4854fc4e4f3dc4578ae732a4aa9990933f47b7d
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Oct 19 23:48:56 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Oct 19 23:52:04 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=d4854fc4
fetch: add docstring
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index debab38a2..76e4636c2 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -433,7 +433,31 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
def fetch(myuris, mysettings, listonly=0, fetchonly=0,
locks_in_subdir=".locks", use_locks=1, try_mirrors=1, digests=None,
allow_missing_digests=True):
- "fetch files. Will use digest file if available."
+ """
+ Fetch files to DISTDIR and also verify digests if they are available.
+
+ @param myuris: Maps each file name to a tuple of available fetch URIs.
+ @type myuris: dict
+ @param mysettings: Portage config instance.
+ @type mysettings: portage.config
+ @param listonly: Only print URIs and do not actually fetch them.
+ @type listonly: bool
+ @param fetchonly: Do not block for files that are locked by a
+ concurrent fetcher process. This means that the function can
+ return successfully *before* all files have been successfully
+ fetched!
+ @type fetchonly: bool
+ @param use_locks: Enable locks. This parameter is ineffective if
+ FEATURES=distlocks is disabled in the portage config!
+ @type use_locks: bool
+ @param digests: Maps each file name to a dict of digest types and values.
+ @type digests: dict
+ @param allow_missing_digests: Enable fetch even if there are no digests
+ available for verification.
+ @type allow_missing_digests: bool
+ @rtype: int
+ @return: 1 if successful, 0 otherwise.
+ """
if not myuris:
return 1
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-10-14 20:45 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-10-14 20:45 UTC (permalink / raw
To: gentoo-commits
commit: 76fd5593d8ddf446f445ba3dd9da18a6e44dc5b3
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 14 20:37:25 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 14 20:38:29 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=76fd5593
get_mirror_url: do not cache after ConfigParserError
Bug: https://bugs.gentoo.org/697566
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 107fba50e..debab38a2 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -415,11 +415,8 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
finally:
f.close()
- try:
- mirror_conf.read_from_file(data)
- except ConfigParserError:
- pass
- except (IOError, UnicodeDecodeError):
+ mirror_conf.read_from_file(data)
+ except (ConfigParserError, IOError, UnicodeDecodeError):
# Do not cache negative results.
cache_path = None
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-10-14 20:35 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-10-14 20:35 UTC (permalink / raw
To: gentoo-commits
commit: 41ae830027648e65f867476a4039e7f51db64673
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 14 20:34:37 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 14 20:35:10 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=41ae8300
get_mirror_url: do not cache negative results
Bug: https://bugs.gentoo.org/697566
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 4e67dff97..107fba50e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -420,7 +420,8 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
except ConfigParserError:
pass
except (IOError, UnicodeDecodeError):
- pass
+ # Do not cache negative results.
+ cache_path = None
cache[mirror_url] = (time.time(), mirror_conf.serialize())
if cache_path is not None:
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-10-14 20:13 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-10-14 20:13 UTC (permalink / raw
To: gentoo-commits
commit: 7c0fd425fd33a75dc8978b5d9ef6d4bd7f8a31bc
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 14 20:10:41 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 14 20:11:40 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=7c0fd425
get_mirror_url: handle UnicodeDecodeError
Fixes: 6a539b7c5163 ("fetch: Support GLEP 75 mirror structure")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 1d5e07260..4e67dff97 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -419,7 +419,7 @@ def get_mirror_url(mirror_url, filename, cache_path=None):
mirror_conf.read_from_file(data)
except ConfigParserError:
pass
- except IOError:
+ except (IOError, UnicodeDecodeError):
pass
cache[mirror_url] = (time.time(), mirror_conf.serialize())
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-10-13 19:50 Michał Górny
0 siblings, 0 replies; 93+ messages in thread
From: Michał Górny @ 2019-10-13 19:50 UTC (permalink / raw
To: gentoo-commits
commit: 0e8d17b56f5b86bd4b66d4720808e8b30c90a0ed
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun Oct 13 11:55:29 2019 +0000
Commit: Michał Górny <mgorny <AT> gentoo <DOT> org>
CommitDate: Sun Oct 13 19:49:49 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=0e8d17b5
fetch: Make FlatLayout.get_filenames() not return directories
Stop returning directories from FlatLayout.get_filenames(). This causes
emirrordist to wrongly presume directories created by new layout
to be distfiles, and causes some noisy errors.
Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Signed-off-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 5d0bc7355..cd204b755 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -35,6 +35,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util:atomic_ofstream',
'portage.util.configparser:SafeConfigParser,read_configs,' +
'ConfigParserError',
+ 'portage.util.install_mask:_raise_exc',
'portage.util._urlopen:urlopen',
)
@@ -269,7 +270,9 @@ class FlatLayout(object):
return filename
def get_filenames(self, distdir):
- return iter(os.listdir(distdir))
+ for dirpath, dirnames, filenames in os.walk(distdir,
+ onerror=_raise_exc):
+ return iter(filenames)
@staticmethod
def verify_args(args):
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-09-03 15:36 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-09-03 15:36 UTC (permalink / raw
To: gentoo-commits
commit: c9c2bcc619c6ff60e452205f94ae26d4afd44d82
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Sep 3 15:34:08 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Sep 3 15:36:23 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=c9c2bcc6
Revert "_doebuild_path: Optimize PATH in ebuild environment."
This reverts commit 7cb7196fe8aa668e76437a8a4500a7bcb9fd3c75.
Since binary and installed packages have persistent PATH
settings in environment.bz2, using realpath on PATH components
can break binary and installed package phases if the symlink
layout is somehow modified. Therefore, the implications of
changes like this need to be considered very carefully.
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 10 +---------
1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 66e294d51..584ff798b 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -5,7 +5,6 @@ from __future__ import unicode_literals
__all__ = ['doebuild', 'doebuild_environment', 'spawn', 'spawnebuild']
-import collections
import grp
import gzip
import errno
@@ -255,14 +254,7 @@ def _doebuild_path(settings, eapi=None):
path.append(x_abs)
path.extend(rootpath)
-
- # Canonicalize paths and avoid duplicates when some directories
- # (e.g. bin and sbin) are merged.
- real_path = collections.OrderedDict()
- for x in path:
- real_path.setdefault(os.path.realpath(x), None)
-
- settings["PATH"] = ":".join(real_path)
+ settings["PATH"] = ":".join(path)
def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
debug=False, use_cache=None, db=None):
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-09-02 20:13 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-09-02 20:13 UTC (permalink / raw
To: gentoo-commits
commit: 7cb7196fe8aa668e76437a8a4500a7bcb9fd3c75
Author: Arfrever Frehtes Taifersar Arahesis <Arfrever <AT> Apache <DOT> Org>
AuthorDate: Mon Sep 2 05:32:09 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Sep 2 20:11:26 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=7cb7196f
_doebuild_path: Optimize PATH in ebuild environment.
Canonicalize paths of directories in PATH.
Avoid duplicates when some directories (e.g. bin and sbin) are merged.
Bug: https://bugs.gentoo.org/693306
Signed-off-by: Arfrever Frehtes Taifersar Arahesis <Arfrever <AT> Apache.Org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 584ff798b..66e294d51 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -5,6 +5,7 @@ from __future__ import unicode_literals
__all__ = ['doebuild', 'doebuild_environment', 'spawn', 'spawnebuild']
+import collections
import grp
import gzip
import errno
@@ -254,7 +255,14 @@ def _doebuild_path(settings, eapi=None):
path.append(x_abs)
path.extend(rootpath)
- settings["PATH"] = ":".join(path)
+
+ # Canonicalize paths and avoid duplicates when some directories
+ # (e.g. bin and sbin) are merged.
+ real_path = collections.OrderedDict()
+ for x in path:
+ real_path.setdefault(os.path.realpath(x), None)
+
+ settings["PATH"] = ":".join(real_path)
def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
debug=False, use_cache=None, db=None):
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-08-20 23:43 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-08-20 23:43 UTC (permalink / raw
To: gentoo-commits
commit: 26c8526293bd93906390d8fc830dc55052fc4d40
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 20 23:41:24 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Aug 20 23:42:53 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=26c85262
doebuild_environment: add newline to PORTAGE_COMPRESSION_COMMAND messages
Fixes: cff2c0149142 ("Support different compressors for binary packages")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 6f980f87d..584ff798b 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -543,7 +543,7 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
compression = _compressors[binpkg_compression]
except KeyError as e:
if binpkg_compression:
- writemsg("Warning: Invalid or unsupported compression method: %s" % e.args[0])
+ writemsg("Warning: Invalid or unsupported compression method: %s\n" % e.args[0])
else:
# Empty BINPKG_COMPRESS disables compression.
mysettings['PORTAGE_COMPRESSION_COMMAND'] = 'cat'
@@ -551,11 +551,11 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
try:
compression_binary = shlex_split(varexpand(compression["compress"], mydict=settings))[0]
except IndexError as e:
- writemsg("Warning: Invalid or unsupported compression method: %s" % e.args[0])
+ writemsg("Warning: Invalid or unsupported compression method: %s\n" % e.args[0])
else:
if find_binary(compression_binary) is None:
missing_package = compression["package"]
- writemsg("Warning: File compression unsupported %s. Missing package: %s" % (binpkg_compression, missing_package))
+ writemsg("Warning: File compression unsupported %s. Missing package: %s\n" % (binpkg_compression, missing_package))
else:
cmd = [varexpand(x, mydict=settings) for x in shlex_split(compression["compress"])]
# Filter empty elements
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-08-14 1:40 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-08-14 1:40 UTC (permalink / raw
To: gentoo-commits
commit: f28d32f298d4b089a2e36bebca2a55b6aeabe2a3
Author: Michał Górny <mgorny <AT> gentoo <DOT> org>
AuthorDate: Sun Aug 11 13:51:35 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Wed Aug 14 01:29:38 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=f28d32f2
Allow ESYSROOT and BROOT in the pkg_setup phase
This follows a recent change to PMS.
Signed-off-by: James Le Cuirot <chewi <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 83a15b370..e0dda54d4 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -2820,12 +2820,13 @@ class config(object):
if not eapi_exports_merge_type(eapi):
mydict.pop("MERGE_TYPE", None)
- src_phase = _phase_func_map.get(phase, '').startswith('src_')
+ src_like_phase = (phase == 'setup' or
+ _phase_func_map.get(phase, '').startswith('src_'))
- if not (src_phase and eapi_attrs.sysroot):
+ if not (src_like_phase and eapi_attrs.sysroot):
mydict.pop("ESYSROOT", None)
- if not (src_phase and eapi_attrs.broot):
+ if not (src_like_phase and eapi_attrs.broot):
mydict.pop("BROOT", None)
# Prefix variables are supported beginning with EAPI 3, or when
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-07-30 7:09 Ulrich Müller
0 siblings, 0 replies; 93+ messages in thread
From: Ulrich Müller @ 2019-07-30 7:09 UTC (permalink / raw
To: gentoo-commits
commit: 4fea794c94932a63942c0722183e1c2dffd6ecd2
Author: Ulrich Müller <ulm <AT> gentoo <DOT> org>
AuthorDate: Mon Jul 29 12:22:57 2019 +0000
Commit: Ulrich Müller <ulm <AT> gentoo <DOT> org>
CommitDate: Tue Jul 30 07:07:10 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=4fea794c
doebuild.py: Override network-sandbox in unpack only for live ebuilds.
Suggested-by: Michał Górny <mgorny <AT> gentoo.org>
Reviewed-by: Zac Medico <zmedico <AT> gentoo.org>
Signed-off-by: Ulrich Müller <ulm <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 67867d33e..6f980f87d 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -115,13 +115,6 @@ _ipc_phases = frozenset([
_global_pid_phases = frozenset([
'config', 'depend', 'preinst', 'prerm', 'postinst', 'postrm'])
-# phases in which networking access is allowed
-_networked_phases = frozenset([
- # for VCS fetching
- "unpack",
- # + for network-bound IPC
-] + list(_ipc_phases))
-
_phase_func_map = {
"config": "pkg_config",
"setup": "pkg_setup",
@@ -156,7 +149,9 @@ def _doebuild_spawn(phase, settings, actionmap=None, **kwargs):
phase in _ipc_phases
kwargs['mountns'] = 'mount-sandbox' in settings.features
kwargs['networked'] = 'network-sandbox' not in settings.features or \
- phase in _networked_phases or \
+ (phase == 'unpack' and \
+ 'live' in settings.configdict['pkg'].get('PROPERTIES', '').split()) or \
+ phase in _ipc_phases or \
'network-sandbox' in settings['PORTAGE_RESTRICT'].split()
kwargs['pidns'] = ('pid-sandbox' in settings.features and
phase not in _global_pid_phases)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-06-18 17:14 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-06-18 17:14 UTC (permalink / raw
To: gentoo-commits
commit: f9f28734257129dd102d16e22ca5a6432243ddc3
Author: Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Mon Jun 17 23:09:23 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Jun 18 17:11:06 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=f9f28734
ebuild: fix QA_SONAME_NO_SYMLINK
misc-functions.sh and ebuild.5 reference QA_SONAME_NO_SYMLINK.
Update doebuild.py to match this name.
Fixes: c9aebae4633a ("_post_src_install_soname_symlinks: fix bug 543818")
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index bea5535bd..67867d33e 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -2263,7 +2263,7 @@ def _post_src_install_soname_symlinks(mysettings, out):
f.close()
metadata = {}
- for k in ("QA_PREBUILT", "QA_NO_SYMLINK"):
+ for k in ("QA_PREBUILT", "QA_SONAME_NO_SYMLINK"):
try:
with io.open(_unicode_encode(os.path.join(
mysettings["PORTAGE_BUILDDIR"],
@@ -2284,14 +2284,14 @@ def _post_src_install_soname_symlinks(mysettings, out):
fnmatch.translate(x.lstrip(os.sep))
for x in portage.util.shlex_split(qa_prebuilt)))
- qa_no_symlink = metadata.get("QA_NO_SYMLINK", "").split()
- if qa_no_symlink:
- if len(qa_no_symlink) > 1:
- qa_no_symlink = "|".join("(%s)" % x for x in qa_no_symlink)
- qa_no_symlink = "^(%s)$" % qa_no_symlink
+ qa_soname_no_symlink = metadata.get("QA_SONAME_NO_SYMLINK", "").split()
+ if qa_soname_no_symlink:
+ if len(qa_soname_no_symlink) > 1:
+ qa_soname_no_symlink = "|".join("(%s)" % x for x in qa_soname_no_symlink)
+ qa_soname_no_symlink = "^(%s)$" % qa_soname_no_symlink
else:
- qa_no_symlink = "^%s$" % qa_no_symlink[0]
- qa_no_symlink = re.compile(qa_no_symlink)
+ qa_soname_no_symlink = "^%s$" % qa_soname_no_symlink[0]
+ qa_soname_no_symlink = re.compile(qa_soname_no_symlink)
libpaths = set(portage.util.getlibpaths(
mysettings["ROOT"], env=mysettings))
@@ -2400,7 +2400,7 @@ def _post_src_install_soname_symlinks(mysettings, out):
continue
if not is_libdir(os.path.dirname(obj)):
continue
- if qa_no_symlink and qa_no_symlink.match(obj.strip(os.sep)) is not None:
+ if qa_soname_no_symlink and qa_soname_no_symlink.match(obj.strip(os.sep)) is not None:
continue
obj_file_path = os.path.join(image_dir, obj.lstrip(os.sep))
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-06-10 19:01 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-06-10 19:01 UTC (permalink / raw
To: gentoo-commits
commit: c8e59b89b3d8f1fea20ec787c65d83becc0070ff
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Jun 10 18:28:56 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Jun 10 18:47:39 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=c8e59b89
config: don't swallow IOError for "packages" files (bug 687814)
Fixes: 55b2e1c0a51d ("Turn a traceback into a graceful error")
Bug: https://bugs.gentoo.org/687814
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index cc2413989..780013bca 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -51,6 +51,7 @@ from portage.util import ensure_dirs, getconfig, grabdict, \
grabdict_package, grabfile, grabfile_package, LazyItemsDict, \
normalize_path, shlex_split, stack_dictlist, stack_dicts, stack_lists, \
writemsg, writemsg_level, _eapi_cache
+from portage.util.install_mask import _raise_exc
from portage.util.path import first_existing
from portage.util._path import exists_raise_eaccess, isdir_raise_eaccess
from portage.versions import catpkgsplit, catsplit, cpv_getkey, _pkg_str
@@ -596,10 +597,8 @@ class config(object):
verify_eapi=True, eapi=x.eapi, eapi_default=None,
allow_build_id=x.allow_build_id)
for x in profiles_complex]
- except IOError as e:
- if e.errno == IsADirectory.errno:
- raise IsADirectory(os.path.join(self.profile_path,
- "packages"))
+ except EnvironmentError as e:
+ _raise_exc(e)
self.packages = tuple(stack_lists(packages_list, incremental=1))
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-01-21 22:20 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-01-21 22:20 UTC (permalink / raw
To: gentoo-commits
commit: e2eb3c8252c2ad154e84cc8803974103e9635eee
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Jan 21 22:11:43 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Jan 21 22:18:51 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e2eb3c82
pid-sandbox: whitelist "depend" phase
Whitelist the "depend" phase for pid-sandbox, since this phase has very
little risk of leaving processes running, and pid-sandbox introduces
additional overhead that would affect egencache performance.
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index f11923595..cf95dc406 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -111,7 +111,8 @@ _ipc_phases = frozenset([
])
# phases which execute in the global PID namespace
-_global_pid_phases = frozenset(['preinst', 'postinst', 'prerm', 'postrm', 'config'])
+_global_pid_phases = frozenset([
+ 'config', 'depend', 'preinst', 'prerm', 'postinst', 'postrm'])
# phases in which networking access is allowed
_networked_phases = frozenset([
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2019-01-01 20:58 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2019-01-01 20:58 UTC (permalink / raw
To: gentoo-commits
commit: 464a65b848eb1344cd1eff3545bed311c01d97ea
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Jan 1 20:55:41 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Jan 1 20:57:54 2019 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=464a65b8
_check_temp_dir: fix message to refer to correct bug 378403
Bug: https://bugs.gentoo.org/378403
Fixes: 8a85160f4833 ("_check_temp_dir: check for 'portage' symlink")
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 47c69967c..2dd458835 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -1305,7 +1305,7 @@ def _check_temp_dir(settings):
msg = _("The 'portage' subdirectory of the directory "
"referenced by the PORTAGE_TMPDIR variable appears to be "
"a symlink. In order to avoid sandbox violations (see bug "
- "#378379), you must adjust PORTAGE_TMPDIR instead of using "
+ "#378403), you must adjust PORTAGE_TMPDIR instead of using "
"the symlink located at '%s'. A suitable PORTAGE_TMPDIR "
"setting would be '%s'.") % \
(os.path.join(tmpdir, "portage"), checkdir)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-12-20 4:29 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-12-20 4:29 UTC (permalink / raw
To: gentoo-commits
commit: 22e8951521558a4488a44f4bdd276ced7c24b950
Author: M. J. Everitt <m.j.everitt <AT> iee <DOT> org>
AuthorDate: Sat Dec 15 01:52:54 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Thu Dec 20 04:24:31 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=22e89515
Provide compatibility support for old PORT_LOGDIR* variable names
Suggested-by: Zac Medico <zmedico <AT> gentoo.org>
Acked-by: Michael Everitt <m.j.everitt <AT> iee.org>
Bug: https://bugs.gentoo.org/668538
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index a68df5807..8de23e2e5 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -155,6 +155,9 @@ class config(object):
_constant_keys = frozenset(['PORTAGE_BIN_PATH', 'PORTAGE_GID',
'PORTAGE_PYM_PATH', 'PORTAGE_PYTHONPATH'])
+ _deprecated_keys = {'PORTAGE_LOGDIR': 'PORT_LOGDIR',
+ 'PORTAGE_LOGDIR_CLEAN': 'PORT_LOGDIR_CLEAN'}
+
_setcpv_aux_keys = ('BDEPEND', 'DEFINED_PHASES', 'DEPEND', 'EAPI', 'HDEPEND',
'INHERITED', 'IUSE', 'REQUIRED_USE', 'KEYWORDS', 'LICENSE', 'PDEPEND',
'PROPERTIES', 'RDEPEND', 'SLOT',
@@ -2653,6 +2656,14 @@ class config(object):
except KeyError:
pass
+ deprecated_key = self._deprecated_keys.get(mykey)
+ if deprecated_key is not None:
+ value = self._getitem(deprecated_key)
+ #warnings.warn(_("Key %s has been renamed to %s. Please ",
+ # "update your configuration") % (deprecated_key, mykey),
+ # UserWarning)
+ return value
+
raise KeyError(mykey)
def get(self, k, x=None):
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-12-07 0:41 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-12-07 0:41 UTC (permalink / raw
To: gentoo-commits
commit: 3ff6d9f92e16cc8b416a23486709adbd5bc13a06
Author: Louis Sautier <sautier.louis <AT> gmail <DOT> com>
AuthorDate: Sun Nov 11 12:12:06 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Dec 7 00:40:05 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=3ff6d9f9
doebuild: add missing whitespace in warning message
Closes: https://github.com/gentoo/portage/pull/380
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 9917ac82c..5da9c9503 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -510,8 +510,8 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
mysettings["PATH"] = p + ":" + mysettings["PATH"]
break
else:
- writemsg(("Warning: %s requested but no masquerade dir"
- + "can be found in /usr/lib*/%s/bin\n") % (m, m))
+ writemsg(("Warning: %s requested but no masquerade dir "
+ "can be found in /usr/lib*/%s/bin\n") % (m, m))
mysettings.features.remove(feature)
if 'MAKEOPTS' not in mysettings:
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-11-19 6:43 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-11-19 6:43 UTC (permalink / raw
To: gentoo-commits
commit: 58eb01022e604b0a538b9d67bd64daa44da9f94a
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Nov 19 06:38:21 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Nov 19 06:42:06 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=58eb0102
Revert "fetch: check DISTDIR write access as fetch user (bug 601252)"
This reverts commit 4264ac4b07c4555eb5db2ab21288dba05a7f4b6d.
Dropping privileges for the access call is not useful by itself,
since lock and unlink calls do not drop privileges yet. We might
decide to drop privileges for the entire fetch function call.
Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 7ec7fe05e..0431e11ea 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -543,7 +543,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
if can_fetch and \
not fetch_to_ro and \
- _spawn_fetch(mysettings, ['test', '-w', mysettings["DISTDIR"]]) != os.EX_OK:
+ not os.access(mysettings["DISTDIR"], os.W_OK):
writemsg(_("!!! No write access to '%s'\n") % mysettings["DISTDIR"],
noiselevel=-1)
can_fetch = False
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-11-09 3:22 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-11-09 3:22 UTC (permalink / raw
To: gentoo-commits
commit: 4264ac4b07c4555eb5db2ab21288dba05a7f4b6d
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Nov 9 03:15:49 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Nov 9 03:21:55 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=4264ac4b
fetch: check DISTDIR write access as fetch user (bug 601252)
Test DISTDIR write access as the fetch user, for correct results
on NFS shares with root_squash enabled.
Bug: https://bugs.gentoo.org/601252
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/fetch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py
index 0431e11ea..7ec7fe05e 100644
--- a/lib/portage/package/ebuild/fetch.py
+++ b/lib/portage/package/ebuild/fetch.py
@@ -543,7 +543,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0,
if can_fetch and \
not fetch_to_ro and \
- not os.access(mysettings["DISTDIR"], os.W_OK):
+ _spawn_fetch(mysettings, ['test', '-w', mysettings["DISTDIR"]]) != os.EX_OK:
writemsg(_("!!! No write access to '%s'\n") % mysettings["DISTDIR"],
noiselevel=-1)
can_fetch = False
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-10-08 21:41 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-10-08 21:41 UTC (permalink / raw
To: gentoo-commits
commit: 01b8e9277b3f6970ea09135def26b70dbf072a16
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 8 02:37:59 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Oct 8 04:29:34 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=01b8e927
_doebuild_path: Respect order defined in ROOTPATH (bug 667662)
Respect the order of paths defined in /etc/env.d/*, so that
packages like llvm can rely on ordering relative to paths
defined in /etc/env.d/50baselayout since baselayout-2.6.
See: https://gitweb.gentoo.org/proj/baselayout.git/commit/?id=277e5b9e55717873b87eb541a95f4f2ae0c60a4d
Bug: https://bugs.gentoo.org/667662
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
lib/portage/package/ebuild/doebuild.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/lib/portage/package/ebuild/doebuild.py b/lib/portage/package/ebuild/doebuild.py
index 941a597e2..621fe7360 100644
--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -212,6 +212,7 @@ def _doebuild_path(settings, eapi=None):
eprefix = portage.const.EPREFIX
prerootpath = [x for x in settings.get("PREROOTPATH", "").split(":") if x]
rootpath = [x for x in settings.get("ROOTPATH", "").split(":") if x]
+ rootpath_set = frozenset(rootpath)
overrides = [x for x in settings.get(
"__PORTAGE_TEST_PATH_OVERRIDE", "").split(":") if x]
@@ -243,7 +244,10 @@ def _doebuild_path(settings, eapi=None):
for prefix in prefixes:
for x in ("usr/local/sbin", "usr/local/bin", "usr/sbin", "usr/bin", "sbin", "bin"):
- path.append(os.path.join(prefix, x))
+ # Respect order defined in ROOTPATH
+ x_abs = os.path.join(prefix, x)
+ if x_abs not in rootpath_set:
+ path.append(x_abs)
path.extend(rootpath)
settings["PATH"] = ":".join(path)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-08-20 23:11 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-08-20 23:11 UTC (permalink / raw
To: gentoo-commits
commit: cbe6f8cac95f5db7b35018a01c56dc3d9c76bb21
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Mon Aug 20 21:54:14 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug 20 22:13:59 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=cbe6f8ca
config: regenerate USE for conditional RESTRICT (bug 664104)
When the features USE state changes due to package.env settings,
regenerate USE for correct evaluation of conditional RESTRICT.
Fixes: 45986341a80c ("Support !test? conditionals in RESTRICT (bug 663278)")
Bug: https://bugs.gentoo.org/664104
lib/portage/package/ebuild/config.py | 39 ++++++++++++++++++------------------
1 file changed, 20 insertions(+), 19 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 9fbf7e8fc..a68df5807 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1666,8 +1666,22 @@ class config(object):
has_changed = True
if has_changed:
+ # This can modify self.features due to package.env settings.
self.reset(keeping_pkg=1)
+ if "test" in self.features:
+ # This is independent of IUSE and RESTRICT, so that the same
+ # value can be shared between packages with different settings,
+ # which is important when evaluating USE conditional RESTRICT.
+ feature_use.append("test")
+
+ feature_use = " ".join(feature_use)
+ if feature_use != self.configdict["features"]["USE"]:
+ # Regenerate USE for evaluation of conditional RESTRICT.
+ self.configdict["features"]["USE"] = feature_use
+ self.reset(keeping_pkg=1)
+ has_changed = True
+
if explicit_iuse is None:
explicit_iuse = frozenset(x.lstrip("+-") for x in iuse.split())
if eapi_attrs.iuse_effective:
@@ -1696,25 +1710,12 @@ class config(object):
else:
restrict_test = "test" in restrict
- pkginternaluse_before = pkginternaluse
- if "test" in self.features:
- # This is independent of IUSE and RESTRICT, so that the same
- # value can be shared between packages with different settings,
- # which is important when evaluating USE conditional RESTRICT
- # above.
- feature_use.append("test")
-
- if restrict_test:
- # Handle it like IUSE="-test", since features USE is
- # independent of RESTRICT.
- pkginternaluse_list.append("-test")
- pkginternaluse = " ".join(pkginternaluse_list)
- self.configdict["pkginternal"]["USE"] = pkginternaluse
-
- feature_use = " ".join(feature_use)
- if (feature_use != self.configdict["features"].get("USE", "") or
- pkginternaluse is not pkginternaluse_before):
- self.configdict["features"]["USE"] = feature_use
+ if restrict_test and "test" in self.features:
+ # Handle it like IUSE="-test", since features USE is
+ # independent of RESTRICT.
+ pkginternaluse_list.append("-test")
+ pkginternaluse = " ".join(pkginternaluse_list)
+ self.configdict["pkginternal"]["USE"] = pkginternaluse
# TODO: can we avoid that?
self.reset(keeping_pkg=1)
has_changed = True
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-08-17 22:35 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-08-17 22:35 UTC (permalink / raw
To: gentoo-commits
commit: ad607b8ecfad3e099cae508ff4d3b5b62cab969f
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Aug 17 11:08:21 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Aug 17 20:12:32 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=ad607b8e
config: default features USE for conditional RESTRICT (bug 663848)
Initialize default features USE state, and reset when appropriate,
so that the "test" flag state is correct for evaluation of !test?
conditionals in RESTRICT.
Fixes: 45986341a80c ("Support !test? conditionals in RESTRICT (bug 663278)")
Bug: https://bugs.gentoo.org/663848
lib/portage/package/ebuild/config.py | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 3b01095d0..9fbf7e8fc 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -259,6 +259,7 @@ class config(object):
self.packages = clone.packages
self.repositories = clone.repositories
self.unpack_dependencies = clone.unpack_dependencies
+ self._default_features_use = clone._default_features_use
self._iuse_effective = clone._iuse_effective
self._iuse_implicit_match = clone._iuse_implicit_match
self._non_user_variables = clone._non_user_variables
@@ -961,6 +962,14 @@ class config(object):
# initialize self.features
self.regenerate()
+ feature_use = []
+ if "test" in self.features:
+ feature_use.append("test")
+ self.configdict["features"]["USE"] = self._default_features_use = " ".join(feature_use)
+ if feature_use:
+ # Regenerate USE so that the initial "test" flag state is
+ # correct for evaluation of !test? conditionals in RESTRICT.
+ self.regenerate()
if unprivileged:
self.features.add('unprivileged')
@@ -1296,7 +1305,7 @@ class config(object):
del self._penv[:]
self.configdict["pkg"].clear()
self.configdict["pkginternal"].clear()
- self.configdict["features"].clear()
+ self.configdict["features"]["USE"] = self._default_features_use
self.configdict["repo"].clear()
self.configdict["defaults"]["USE"] = \
" ".join(self.make_defaults_use)
@@ -1598,7 +1607,7 @@ class config(object):
has_changed = True
# Prevent stale features USE from corrupting the evaluation
# of USE conditional RESTRICT.
- self.configdict["features"]["USE"] = ""
+ self.configdict["features"]["USE"] = self._default_features_use
self._penv = []
cpdict = self._penvdict.get(cp)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-08-12 2:31 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-08-12 2:31 UTC (permalink / raw
To: gentoo-commits
commit: 45986341a80cfb01dad470f56f02b210b3ebf753
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Aug 11 21:04:05 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Aug 12 00:40:32 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=45986341
Support !test? conditionals in RESTRICT (bug 663278)
Since RESTRICT="!test? ( test )" can be very useful within the context
of bug 663278, pass an appropriate uselist parameter to the RESTRICT
use_reduce call. Make self.configdict["features"]["USE"] independent
of IUSE and RESTRICT, so that the same value can be shared between
packages with different settings, which is important when evaluating
USE conditional RESTRICT. When the evaluated value of RESTRICT contains
"test", handle it like IUSE="-test", since features USE is independent
of RESTRICT.
Bug: https://bugs.gentoo.org/663278
lib/portage/package/ebuild/config.py | 40 +++++++++++++++++++++++-------------
1 file changed, 26 insertions(+), 14 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 220fa31bb..3b01095d0 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1457,6 +1457,7 @@ class config(object):
cp = cpv_getkey(mycpv)
cpv_slot = self.mycpv
pkginternaluse = ""
+ pkginternaluse_list = []
feature_use = []
iuse = ""
pkg_configdict = self.configdict["pkg"]
@@ -1513,13 +1514,12 @@ class config(object):
cpv_slot = self.mycpv
else:
cpv_slot = pkg
- pkginternaluse = []
for x in iuse.split():
if x.startswith("+"):
- pkginternaluse.append(x[1:])
+ pkginternaluse_list.append(x[1:])
elif x.startswith("-"):
- pkginternaluse.append(x)
- pkginternaluse = " ".join(pkginternaluse)
+ pkginternaluse_list.append(x)
+ pkginternaluse = " ".join(pkginternaluse_list)
eapi_attrs = _get_eapi_attrs(eapi)
@@ -1596,6 +1596,9 @@ class config(object):
# regenerate() call in order to ensure that self.features
# is accurate.
has_changed = True
+ # Prevent stale features USE from corrupting the evaluation
+ # of USE conditional RESTRICT.
+ self.configdict["features"]["USE"] = ""
self._penv = []
cpdict = self._penvdict.get(cp)
@@ -1675,24 +1678,33 @@ class config(object):
restrict = use_reduce(raw_restrict,
uselist=built_use, flat=True)
else:
- # Use matchnone=True to ignore USE conditional parts
- # of RESTRICT, since we want to know whether to mask
- # the "test" flag _before_ we know the USE values
- # that would be needed to evaluate the USE
- # conditionals (see bug #273272).
restrict = use_reduce(raw_restrict,
- matchnone=True, flat=True)
+ uselist=frozenset(x for x in self['USE'].split()
+ if x in explicit_iuse or iuse_implicit_match(x)),
+ flat=True)
except PortageException:
pass
else:
restrict_test = "test" in restrict
- if not restrict_test and ("test" in explicit_iuse or iuse_implicit_match("test")):
- if "test" in self.features:
- feature_use.append("test")
+ pkginternaluse_before = pkginternaluse
+ if "test" in self.features:
+ # This is independent of IUSE and RESTRICT, so that the same
+ # value can be shared between packages with different settings,
+ # which is important when evaluating USE conditional RESTRICT
+ # above.
+ feature_use.append("test")
+
+ if restrict_test:
+ # Handle it like IUSE="-test", since features USE is
+ # independent of RESTRICT.
+ pkginternaluse_list.append("-test")
+ pkginternaluse = " ".join(pkginternaluse_list)
+ self.configdict["pkginternal"]["USE"] = pkginternaluse
feature_use = " ".join(feature_use)
- if feature_use != self.configdict["features"].get("USE", ""):
+ if (feature_use != self.configdict["features"].get("USE", "") or
+ pkginternaluse is not pkginternaluse_before):
self.configdict["features"]["USE"] = feature_use
# TODO: can we avoid that?
self.reset(keeping_pkg=1)
^ permalink raw reply related [flat|nested] 93+ messages in thread* [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/
@ 2018-08-11 21:06 Zac Medico
0 siblings, 0 replies; 93+ messages in thread
From: Zac Medico @ 2018-08-11 21:06 UTC (permalink / raw
To: gentoo-commits
commit: abf6f36a2785671f183f5f898f03c29cd5a915b9
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Aug 11 19:48:01 2018 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Aug 11 21:05:20 2018 +0000
URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=abf6f36a
Make features USE respect RESTRICT=test (bug 663278)
Make RESTRICT=test prevent the "test" USE flag from being added to
features USE flags when FEATURES=test is enabled, in order to preserve
default behavior for ebuilds that set RESTRICT=test. The code that sets
the restrict_test variable in the setcpv method must execute earlier
now, but the logic is unchanged. Note that it is still possible to
enable USE=test for ebuilds that set RESTRICT=test, but FEATURES=test
will not do it, so it will only be triggered by an explicit USE=test
setting by the user or profile.
Fixes: 8c5598c1af2c ("Replace implicit {FEATURES->USE}=test forcing with USE default")
Bug: https://bugs.gentoo.org/663278
Reviewed-by: Michał Górny <mgorny <AT> gentoo.org>
lib/portage/package/ebuild/config.py | 50 ++++++++++++++++++------------------
1 file changed, 25 insertions(+), 25 deletions(-)
diff --git a/lib/portage/package/ebuild/config.py b/lib/portage/package/ebuild/config.py
index 3e0081829..220fa31bb 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -1663,7 +1663,31 @@ class config(object):
else:
iuse_implicit_match = self._iuse_implicit_match
- if "test" in explicit_iuse or iuse_implicit_match("test"):
+ if pkg is None:
+ raw_restrict = pkg_configdict.get("RESTRICT")
+ else:
+ raw_restrict = pkg._raw_metadata["RESTRICT"]
+
+ restrict_test = False
+ if raw_restrict:
+ try:
+ if built_use is not None:
+ restrict = use_reduce(raw_restrict,
+ uselist=built_use, flat=True)
+ else:
+ # Use matchnone=True to ignore USE conditional parts
+ # of RESTRICT, since we want to know whether to mask
+ # the "test" flag _before_ we know the USE values
+ # that would be needed to evaluate the USE
+ # conditionals (see bug #273272).
+ restrict = use_reduce(raw_restrict,
+ matchnone=True, flat=True)
+ except PortageException:
+ pass
+ else:
+ restrict_test = "test" in restrict
+
+ if not restrict_test and ("test" in explicit_iuse or iuse_implicit_match("test")):
if "test" in self.features:
feature_use.append("test")
@@ -1721,30 +1745,6 @@ class config(object):
self.configdict["env"].addLazySingleton(
"PORTAGE_IUSE", _lazy_iuse_regex, portage_iuse)
- if pkg is None:
- raw_restrict = pkg_configdict.get("RESTRICT")
- else:
- raw_restrict = pkg._raw_metadata["RESTRICT"]
-
- restrict_test = False
- if raw_restrict:
- try:
- if built_use is not None:
- restrict = use_reduce(raw_restrict,
- uselist=built_use, flat=True)
- else:
- # Use matchnone=True to ignore USE conditional parts
- # of RESTRICT, since we want to know whether to mask
- # the "test" flag _before_ we know the USE values
- # that would be needed to evaluate the USE
- # conditionals (see bug #273272).
- restrict = use_reduce(raw_restrict,
- matchnone=True, flat=True)
- except PortageException:
- pass
- else:
- restrict_test = "test" in restrict
-
ebuild_force_test = not restrict_test and \
self.get("EBUILD_FORCE_TEST") == "1"
^ permalink raw reply related [flat|nested] 93+ messages in thread
end of thread, other threads:[~2024-08-07 14:39 UTC | newest]
Thread overview: 93+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-10-10 7:58 [gentoo-commits] proj/portage:master commit in: lib/portage/package/ebuild/ Zac Medico
-- strict thread matches above, loose matches on Subject: below --
2024-08-07 14:39 Zac Medico
2024-07-19 5:41 Sam James
2024-06-17 0:25 Sam James
2024-06-17 0:20 Sam James
2024-03-24 22:20 Zac Medico
2023-12-20 14:04 Sam James
2023-12-20 14:04 Sam James
2023-12-20 14:04 Sam James
2023-12-10 22:01 Sam James
2023-10-22 22:46 Zac Medico
2023-10-16 5:15 Zac Medico
2023-10-13 10:33 Sam James
2023-10-13 10:19 Sam James
2023-10-13 10:19 Sam James
2023-08-19 15:05 Sam James
2023-06-14 19:23 Mike Gilbert
2023-06-14 19:23 Mike Gilbert
2023-02-17 0:53 Sam James
2023-01-27 8:38 Ulrich Müller
2023-01-02 20:45 Mike Gilbert
2022-07-12 23:33 Sam James
2022-06-12 18:25 Sam James
2021-12-11 2:54 Sam James
2021-12-11 2:54 Sam James
2021-11-15 8:34 Michał Górny
2021-11-08 22:37 Zac Medico
2021-10-28 5:00 Sam James
2021-10-28 4:52 Sam James
2021-10-03 19:31 Zac Medico
2021-09-28 11:25 Michał Górny
2021-09-28 11:19 Michał Górny
2021-09-28 7:21 Zac Medico
2021-09-27 20:51 Michał Górny
2021-09-08 10:17 Michał Górny
2021-08-05 8:47 Michał Górny
2021-06-20 18:54 Zac Medico
2021-06-05 18:08 Zac Medico
2021-06-05 18:08 Zac Medico
2021-05-31 19:54 Michał Górny
2021-05-24 5:25 Zac Medico
2021-02-25 9:33 Zac Medico
2021-02-24 15:14 Zac Medico
2021-02-22 5:32 Zac Medico
2021-01-17 13:15 Zac Medico
2021-01-10 3:24 Zac Medico
2020-11-02 1:34 Zac Medico
2020-09-14 7:23 Zac Medico
2020-08-03 21:42 Zac Medico
2020-08-03 21:42 Zac Medico
2020-08-03 19:30 Zac Medico
2020-08-03 19:30 Zac Medico
2020-08-03 19:30 Zac Medico
2020-05-31 23:58 Mike Gilbert
2020-05-31 21:17 Mike Gilbert
2020-05-31 20:34 Mike Gilbert
2020-05-18 19:23 Michał Górny
2020-05-06 6:37 Michał Górny
2020-03-23 1:49 Zac Medico
2020-03-23 1:40 Zac Medico
2020-03-15 1:24 Zac Medico
2020-03-02 16:53 Zac Medico
2020-03-02 4:48 Zac Medico
2020-02-15 20:44 Zac Medico
2020-02-15 20:28 Zac Medico
2019-12-09 7:03 Zac Medico
2019-10-29 1:17 Zac Medico
2019-10-24 19:31 Zac Medico
2019-10-21 17:49 Zac Medico
2019-10-20 9:26 Michał Górny
2019-10-19 23:52 Zac Medico
2019-10-14 20:45 Zac Medico
2019-10-14 20:35 Zac Medico
2019-10-14 20:13 Zac Medico
2019-10-13 19:50 Michał Górny
2019-09-03 15:36 Zac Medico
2019-09-02 20:13 Zac Medico
2019-08-20 23:43 Zac Medico
2019-08-14 1:40 Zac Medico
2019-07-30 7:09 Ulrich Müller
2019-06-18 17:14 Zac Medico
2019-06-10 19:01 Zac Medico
2019-01-21 22:20 Zac Medico
2019-01-01 20:58 Zac Medico
2018-12-20 4:29 Zac Medico
2018-12-07 0:41 Zac Medico
2018-11-19 6:43 Zac Medico
2018-11-09 3:22 Zac Medico
2018-10-08 21:41 Zac Medico
2018-08-20 23:11 Zac Medico
2018-08-17 22:35 Zac Medico
2018-08-12 2:31 Zac Medico
2018-08-11 21:06 Zac Medico
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox