[gentoo-commits] proj/portage:master commit in: lib/portage/util/_dyn_libs/

[gentoo-commits] proj/portage:master commit in: lib/portage/util/_dyn_libs/

[Zac Medico]

commit:     1120e6f62feb1aa0d3202beb00c01537d352f420
Author:     Benda Xu <heroxbd <AT> gentoo <DOT> org>
AuthorDate: Sat Aug 10 02:26:12 2019 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Aug 11 18:07:43 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=1120e6f6

lib/p/util/_dyn_libs/LinkageMapELF.py: get dep graph from EROOT.

On Prefix, the preserve-libs feature should search for shared
libraries consumers from EROOT instead of ROOT.

Bug: https://bugs.gentoo.org/646090
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/util/_dyn_libs/LinkageMapELF.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/portage/util/_dyn_libs/LinkageMapELF.py b/lib/portage/util/_dyn_libs/LinkageMapELF.py
index a063621c1..92a50b444 100644
--- a/lib/portage/util/_dyn_libs/LinkageMapELF.py
+++ b/lib/portage/util/_dyn_libs/LinkageMapELF.py
@@ -1,4 +1,4 @@
-# Copyright 1998-2016 Gentoo Foundation
+# Copyright 1998-2019 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import errno
@@ -222,7 +222,8 @@ class LinkageMapELF(object):
 		root = self._root
 		root_len = len(root) - 1
 		self._clear_cache()
-		self._defpath.update(getlibpaths(self._root, env=self._dbapi.settings))
+		self._defpath.update(getlibpaths(self._dbapi.settings['EROOT'],
+			env=self._dbapi.settings))
 		libs = self._libs
 		obj_properties = self._obj_properties
 

[gentoo-commits] proj/portage:master commit in: lib/portage/util/_dyn_libs/

[Zac Medico]

commit:     0edb0cd52e25a3ce168f712258ded9ccc6a5bee4
Author:     Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Feb  8 19:43:27 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sat Feb  8 22:49:24 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=0edb0cd5

preserve-libs: generate implicit rpath for bundled libs (bug 705736)

In order to account for internal library resolution which a package
may implement (useful at least for handling of bundled libraries),
generate implicit runpath entries for any needed sonames which are
provided by the same owner package.

For packages that have bundled versions of system libraries, this will
prevent preserve-libs from unecessarily preserving system libraries as
reported in bug 705736. For the www-client/firefox-bin-72.0.2 package,
this adds an implicit /opt/firefox runpath entry to 15 files.

Bug: https://bugs.gentoo.org/705736
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/util/_dyn_libs/LinkageMapELF.py | 63 ++++++++++++++++++++++++-----
 1 file changed, 54 insertions(+), 9 deletions(-)

diff --git a/lib/portage/util/_dyn_libs/LinkageMapELF.py b/lib/portage/util/_dyn_libs/LinkageMapELF.py
index 92a50b444..70bec116a 100644
--- a/lib/portage/util/_dyn_libs/LinkageMapELF.py
+++ b/lib/portage/util/_dyn_libs/LinkageMapELF.py
@@ -1,7 +1,9 @@
-# Copyright 1998-2019 Gentoo Authors
+# Copyright 1998-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
+import collections
 import errno
+import itertools
 import logging
 import subprocess
 import sys
@@ -14,6 +16,7 @@ from portage import _unicode_encode
 from portage.cache.mappings import slot_dict_class
 from portage.const import EPREFIX
 from portage.dep.soname.multilib_category import compute_multilib_category
+from portage.dep.soname.SonameAtom import SonameAtom
 from portage.exception import CommandNotFound, InvalidData
 from portage.localization import _
 from portage.util import getlibpaths
@@ -328,8 +331,13 @@ class LinkageMapELF(object):
 		# Share identical frozenset instances when available,
 		# in order to conserve memory.
 		frozensets = {}
+		owner_entries = collections.defaultdict(list)
 
-		for owner, location, l in lines:
+		while True:
+			try:
+				owner, location, l = lines.pop()
+			except IndexError:
+				break
 			l = l.rstrip("\n")
 			if not l:
 				continue
@@ -352,18 +360,55 @@ class LinkageMapELF(object):
 			# exists, map e_machine (entry.arch) to an approximate
 			# multilib category. If all else fails, use e_machine, just
 			# as older versions of portage did.
-			arch = entry.multilib_category
-			if arch is None:
-				arch = _approx_multilib_categories.get(
+			if entry.multilib_category is None:
+				entry.multilib_category = _approx_multilib_categories.get(
 					entry.arch, entry.arch)
 
-			obj = entry.filename
-			soname = entry.soname
+			entry.filename = normalize_path(entry.filename)
 			expand = {"ORIGIN": os.path.dirname(entry.filename)}
-			path = frozenset(normalize_path(
+			entry.runpaths = frozenset(normalize_path(
 				varexpand(x, expand, error_leader=lambda: "%s: " % location))
 				for x in entry.runpaths)
-			path = frozensets.setdefault(path, path)
+			entry.runpaths = frozensets.setdefault(entry.runpaths, entry.runpaths)
+			owner_entries[owner].append(entry)
+
+		# In order to account for internal library resolution which a package
+		# may implement (useful at least for handling of bundled libraries),
+		# generate implicit runpath entries for any needed sonames which are
+		# provided by the same owner package.
+		for owner, entries in owner_entries.items():
+			if owner is None:
+				continue
+
+			providers = {}
+			for entry in entries:
+				if entry.soname:
+					providers[SonameAtom(entry.multilib_category, entry.soname)] = entry
+
+			for entry in entries:
+				implicit_runpaths = []
+				for soname in entry.needed:
+					soname_atom = SonameAtom(entry.multilib_category, soname)
+					provider = providers.get(soname_atom)
+					if provider is None:
+						continue
+					provider_dir = os.path.dirname(provider.filename)
+					if provider_dir not in entry.runpaths:
+						implicit_runpaths.append(provider_dir)
+
+				if implicit_runpaths:
+					entry.runpaths = frozenset(
+						itertools.chain(entry.runpaths, implicit_runpaths))
+					entry.runpaths = frozensets.setdefault(
+						entry.runpaths, entry.runpaths)
+
+		for owner, entry in ((owner, entry)
+			for (owner, entries) in owner_entries.items()
+			for entry in entries):
+			arch = entry.multilib_category
+			obj = entry.filename
+			soname = entry.soname
+			path = entry.runpaths
 			needed = frozenset(entry.needed)
 
 			needed = frozensets.setdefault(needed, needed)

[gentoo-commits] proj/portage:master commit in: lib/portage/util/_dyn_libs/

[Zac Medico]

commit:     3c9fdead607e61e420f5690ed58205d957d2acef
Author:     Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug  3 19:05:49 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug  3 19:21:51 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=3c9fdead

lib/portage/util/_dyn_libs/NeededEntry.py: drop unused-import

* Drop unused-import
* Update copyright

Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/util/_dyn_libs/NeededEntry.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/lib/portage/util/_dyn_libs/NeededEntry.py b/lib/portage/util/_dyn_libs/NeededEntry.py
index 20dc2f779..77ced0f43 100644
--- a/lib/portage/util/_dyn_libs/NeededEntry.py
+++ b/lib/portage/util/_dyn_libs/NeededEntry.py
@@ -1,8 +1,6 @@
-# Copyright 2015 Gentoo Foundation
+# Copyright 2015-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
-
-from portage import _encodings, _unicode_encode
 from portage.exception import InvalidData
 from portage.localization import _
 

[gentoo-commits] proj/portage:master commit in: lib/portage/util/_dyn_libs/

[Zac Medico]

commit:     62002b7b464e919c701ead4dbdde0d83d4ea229a
Author:     Aaron Bauman <bman <AT> gentoo <DOT> org>
AuthorDate: Mon Aug  3 22:43:20 2020 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Mon Aug  3 23:28:03 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=62002b7b

lib/portage/util/_dyn_libs/LinkageMapELF.py: fix whitespace

Signed-off-by: Aaron Bauman <bman <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 lib/portage/util/_dyn_libs/LinkageMapELF.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portage/util/_dyn_libs/LinkageMapELF.py b/lib/portage/util/_dyn_libs/LinkageMapELF.py
index 954a956c6..930d78587 100644
--- a/lib/portage/util/_dyn_libs/LinkageMapELF.py
+++ b/lib/portage/util/_dyn_libs/LinkageMapELF.py
@@ -144,7 +144,7 @@ class LinkageMapELF:
 				_unicode_encode(obj,
 					encoding=_encodings['merge'], errors='strict')
 			except UnicodeEncodeError:
-				# The package appears to have been merged with a 
+				# The package appears to have been merged with a
 				# different value of sys.getfilesystemencoding(),
 				# so fall back to utf_8 if appropriate.
 				try:
@@ -249,7 +249,7 @@ class LinkageMapELF:
 			if can_lock:
 				self._dbapi.unlock()
 
-		# have to call scanelf for preserved libs here as they aren't 
+		# have to call scanelf for preserved libs here as they aren't
 		# registered in NEEDED.ELF.2 files
 		plibs = {}
 		if preserve_paths is not None:

[gentoo-commits] proj/portage:master commit in: lib/portage/util/_dyn_libs/

[Sam James]

commit:     cba2156dba89a22f2858238013469b4d80208854
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Thu Oct 28 04:40:09 2021 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Thu Oct 28 04:52:18 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=cba2156d

portage/util/_dyn_libs/dyn_libs.py: add helper

Needed for binpkg/image verification commits coming...

Bug: https://bugs.gentoo.org/811462
Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/util/_dyn_libs/dyn_libs.py | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/lib/portage/util/_dyn_libs/dyn_libs.py b/lib/portage/util/_dyn_libs/dyn_libs.py
new file mode 100644
index 000000000..2a014812c
--- /dev/null
+++ b/lib/portage/util/_dyn_libs/dyn_libs.py
@@ -0,0 +1,23 @@
+# Copyright 2021 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+import glob
+
+from portage.output import colorize
+
+
+def check_dyn_libs_inconsistent(directory, provides):
+    """Checks directory for whether any dynamic libraries were installed and
+    if PROVIDES corresponds."""
+
+    # Let's check if we've got inconsistent results.
+    # If we're installing dynamic libraries (.so files), we should
+    # really have a PROVIDES.
+    # (This is a complementary check at the point of ingestion for the
+    # creation check in doebuild.py)
+    # Note: we could check a non-empty PROVIDES against the list of .sos,
+    # but this doesn't gain us anything. We're interested in failure
+    # to properly parse the installed files at all, which should really
+    # be a global problem (e.g. bug #811462)
+    installed_dynlibs = glob.glob(directory + "/**/*.so", recursive=True)
+    return installed_dynlibs and not provides

[gentoo-commits] proj/portage:master commit in: lib/portage/util/_dyn_libs/

[Sam James]

commit:     dcf6de8aa115cfe1ca5474b3c98efa8f75f8f47b
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Thu Oct 28 04:59:20 2021 +0000
Commit:     Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Thu Oct 28 05:00:10 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=dcf6de8a

lib/portage/util/_dyn_libs/dyn_libs.py: drop unused import

Signed-off-by: Sam James <sam <AT> gentoo.org>

 lib/portage/util/_dyn_libs/dyn_libs.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/lib/portage/util/_dyn_libs/dyn_libs.py b/lib/portage/util/_dyn_libs/dyn_libs.py
index 2a014812c..34b4bc926 100644
--- a/lib/portage/util/_dyn_libs/dyn_libs.py
+++ b/lib/portage/util/_dyn_libs/dyn_libs.py
@@ -3,8 +3,6 @@
 
 import glob
 
-from portage.output import colorize
-
 
 def check_dyn_libs_inconsistent(directory, provides):
     """Checks directory for whether any dynamic libraries were installed and

[gentoo-commits] proj/portage:master commit in: lib/portage/util/_dyn_libs/

[Mike Gilbert]

commit:     3b3503bb2769b390e22a4fea42fa56fe4db4ddec
Author:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
AuthorDate: Tue Nov 23 22:39:24 2021 +0000
Commit:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Tue Nov 23 23:22:32 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=3b3503bb

portage/util/_dyn_libs/dyn_libs.py: fix symlink recursion issue

Python's glob.glob() function follows symlinks recursively. Use
os.walk() instead.

Also re-order the operands to short-circuit the directory walk when
PROVIDES is not empty.

Bug: https://bugs.gentoo.org/826982
Fixes: cba2156dba89a22f2858238013469b4d80208854
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>

 lib/portage/util/_dyn_libs/dyn_libs.py | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/lib/portage/util/_dyn_libs/dyn_libs.py b/lib/portage/util/_dyn_libs/dyn_libs.py
index 34b4bc926..ee28e8839 100644
--- a/lib/portage/util/_dyn_libs/dyn_libs.py
+++ b/lib/portage/util/_dyn_libs/dyn_libs.py
@@ -1,7 +1,15 @@
 # Copyright 2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
-import glob
+import os
+
+
+def installed_dynlibs(directory):
+    for _dirpath, _dirnames, filenames in os.walk(directory):
+        for filename in filenames:
+            if filename.endswith(".so"):
+                return True
+    return False
 
 
 def check_dyn_libs_inconsistent(directory, provides):
@@ -17,5 +25,4 @@ def check_dyn_libs_inconsistent(directory, provides):
     # but this doesn't gain us anything. We're interested in failure
     # to properly parse the installed files at all, which should really
     # be a global problem (e.g. bug #811462)
-    installed_dynlibs = glob.glob(directory + "/**/*.so", recursive=True)
-    return installed_dynlibs and not provides
+    return not provides and installed_dynlibs(directory)