public inbox for gentoo-portage-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-portage-dev] [PATCH 0/4] {,PKG_}INSTALL_MASK support for exclusions (bug 651214)
@ 2018-03-27  9:13 Zac Medico
  2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 1/4] {,PKG_}INSTALL_MASK: record value in vardb Zac Medico
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Zac Medico @ 2018-03-27  9:13 UTC (permalink / raw
  To: gentoo-portage-dev; +Cc: Michał Górny, Zac Medico

This patch series adds {,PKG_}INSTALL_MASK support for exclusions.
This includes a python implementation of INSTALL_MASK, with
matching performed by fnmatch code from these portage-mgorny
commits:

https://github.com/mgorny/portage/commit/f5ac3af3216d209618a2f232b4bf720bc8b520ad
https://github.com/mgorny/portage/commit/346da88d931de55cea1ab9657df30e26603469a6

I have specifically excluded changes related to bug 364633, since
CONTENTS modifications add a significant amount of complexity which
has nothing to do with exclusion support.

Michał Górny (1):
  {,PKG_}INSTALL_MASK: Support exclusions (bug 651214)

Zac Medico (3):
  {,PKG_}INSTALL_MASK: record value in vardb
  EbuildPhase: add PackagePhase class for PKG_INSTALL_MASK
  {,PKG_}INSTALL_MASK: python implementation

 bin/misc-functions.sh            |  96 +--------------------------------
 bin/phase-functions.sh           |  12 ++++-
 pym/_emerge/EbuildPhase.py       |   7 +++
 pym/_emerge/PackagePhase.py      |  91 ++++++++++++++++++++++++++++++++
 pym/portage/dbapi/vartree.py     |  24 ++++++---
 pym/portage/util/install_mask.py | 111 +++++++++++++++++++++++++++++++++++++++
 6 files changed, 240 insertions(+), 101 deletions(-)
 create mode 100644 pym/_emerge/PackagePhase.py
 create mode 100644 pym/portage/util/install_mask.py

Bug: https://bugs.gentoo.org/651214
-- 
2.13.6



^ permalink raw reply	[flat|nested] 5+ messages in thread

* [gentoo-portage-dev] [PATCH 1/4] {,PKG_}INSTALL_MASK: record value in vardb
  2018-03-27  9:13 [gentoo-portage-dev] [PATCH 0/4] {,PKG_}INSTALL_MASK support for exclusions (bug 651214) Zac Medico
@ 2018-03-27  9:13 ` Zac Medico
  2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 2/4] EbuildPhase: add PackagePhase class for PKG_INSTALL_MASK Zac Medico
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Zac Medico @ 2018-03-27  9:13 UTC (permalink / raw
  To: gentoo-portage-dev; +Cc: Michał Górny, Zac Medico

Also, skip preinst_mask phase when INSTALL_MASK is empty.
---
 bin/misc-functions.sh  |  8 --------
 bin/phase-functions.sh | 12 +++++++++++-
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index 7643af7b5..742ce40d2 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -375,14 +375,6 @@ preinst_mask() {
 	# in there in case any tools were built with -pg in CFLAGS.
 	cd "${T}"
 
-	# remove man pages, info pages, docs if requested
-	local f
-	for f in man info doc; do
-		if has no${f} $FEATURES; then
-			INSTALL_MASK="${INSTALL_MASK} /usr/share/${f}"
-		fi
-	done
-
 	install_mask "${ED}" "${INSTALL_MASK}"
 
 	# remove share dir if unnessesary
diff --git a/bin/phase-functions.sh b/bin/phase-functions.sh
index 3aae3ef56..bdae68f79 100644
--- a/bin/phase-functions.sh
+++ b/bin/phase-functions.sh
@@ -660,13 +660,23 @@ __dyn_install() {
 	cd "${PORTAGE_BUILDDIR}"/build-info
 	set -f
 	local f x
+
+	# remove man pages, info pages, docs if requested
+	for f in man info doc; do
+		if has no${f} ${FEATURES} && \
+			! has "/usr/share/${f}" ${INSTALL_MASK}; then
+			INSTALL_MASK+=" /usr/share/${f}"
+		fi
+	done
+
 	IFS=$' \t\n\r'
 	for f in CATEGORY DEFINED_PHASES FEATURES INHERITED IUSE \
 		PF PKGUSE SLOT KEYWORDS HOMEPAGE DESCRIPTION \
 		ASFLAGS CBUILD CC CFLAGS CHOST CTARGET CXX \
 		CXXFLAGS EXTRA_ECONF EXTRA_EINSTALL EXTRA_MAKE \
 		LDFLAGS LIBCFLAGS LIBCXXFLAGS QA_CONFIGURE_OPTIONS \
-		QA_DESKTOP_FILE QA_PREBUILT PROVIDES_EXCLUDE REQUIRES_EXCLUDE ; do
+		QA_DESKTOP_FILE QA_PREBUILT PROVIDES_EXCLUDE REQUIRES_EXCLUDE \
+		INSTALL_MASK PKG_INSTALL_MASK; do
 
 		x=$(echo -n ${!f})
 		[[ -n $x ]] && echo "$x" > $f
-- 
2.13.6



^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [gentoo-portage-dev] [PATCH 2/4] EbuildPhase: add PackagePhase class for PKG_INSTALL_MASK
  2018-03-27  9:13 [gentoo-portage-dev] [PATCH 0/4] {,PKG_}INSTALL_MASK support for exclusions (bug 651214) Zac Medico
  2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 1/4] {,PKG_}INSTALL_MASK: record value in vardb Zac Medico
@ 2018-03-27  9:13 ` Zac Medico
  2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 3/4] {,PKG_}INSTALL_MASK: python implementation Zac Medico
  2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 4/4] {,PKG_}INSTALL_MASK: Support exclusions (bug 651214) Zac Medico
  3 siblings, 0 replies; 5+ messages in thread
From: Zac Medico @ 2018-03-27  9:13 UTC (permalink / raw
  To: gentoo-portage-dev; +Cc: Michał Górny, Zac Medico

The PackagePhase class will be a convient place to invoke
a python implementation of PKG_INSTALL_MASK.
---
 bin/misc-functions.sh       | 23 ++++---------
 pym/_emerge/EbuildPhase.py  |  7 ++++
 pym/_emerge/PackagePhase.py | 80 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 93 insertions(+), 17 deletions(-)
 create mode 100644 pym/_emerge/PackagePhase.py

diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index 742ce40d2..4fe6ead17 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -497,29 +497,20 @@ preinst_selinux_labels() {
 }
 
 __dyn_package() {
-	local PROOT
 
 	if ! ___eapi_has_prefix_variables; then
-		local EPREFIX= ED=${D}
+		local EPREFIX=
 	fi
 
 	# Make sure $PWD is not ${D} so that we don't leave gmon.out files
 	# in there in case any tools were built with -pg in CFLAGS.
-
 	cd "${T}" || die
 
 	if [[ -n ${PKG_INSTALL_MASK} ]] ; then
-		PROOT=${T}/packaging/
-		# make a temporary copy of ${D} so that any modifications we do that
-		# are binpkg specific, do not influence the actual installed image.
-		rm -rf "${PROOT}" || die "failed removing stale package tree"
-		cp -pPR $(cp --help | grep -qs -e-l && echo -l) \
-			"${D}" "${PROOT}" \
-			|| die "failed creating packaging tree"
-
-		install_mask "${PROOT%/}${EPREFIX}/" "${PKG_INSTALL_MASK}"
-	else
-		PROOT=${D}
+		# The caller makes ${D} refer to a temporary copy in this
+		# case, so that this does not mask files from the normal
+		# install image.
+		install_mask "${D%/}${EPREFIX}/" "${PKG_INSTALL_MASK}"
 	fi
 
 	local tar_options=""
@@ -533,7 +524,7 @@ __dyn_package() {
 	mkdir -p "${PORTAGE_BINPKG_TMPFILE%/*}" || die "mkdir failed"
 	[ -z "${PORTAGE_COMPRESSION_COMMAND}" ] && \
         die "PORTAGE_COMPRESSION_COMMAND is unset"
-	tar $tar_options -cf - $PORTAGE_BINPKG_TAR_OPTS -C "${PROOT}" . | \
+	tar $tar_options -cf - $PORTAGE_BINPKG_TAR_OPTS -C "${D}" . | \
 		$PORTAGE_COMPRESSION_COMMAND -c > "$PORTAGE_BINPKG_TMPFILE"
 	assert "failed to pack binary package: '$PORTAGE_BINPKG_TMPFILE'"
 	PYTHONPATH=${PORTAGE_PYTHONPATH:-${PORTAGE_PYM_PATH}} \
@@ -555,8 +546,6 @@ __dyn_package() {
 		echo ${md5_hash} > "${PORTAGE_BUILDDIR}"/build-info/BINPKGMD5
 	__vecho ">>> Done."
 
-	# cleanup our temp tree
-	[[ -n ${PKG_INSTALL_MASK} ]] && rm -rf "${PROOT}"
 	cd "${PORTAGE_BUILDDIR}"
 	>> "$PORTAGE_BUILDDIR/.packaged" || \
 		die "Failed to create $PORTAGE_BUILDDIR/.packaged"
diff --git a/pym/_emerge/EbuildPhase.py b/pym/_emerge/EbuildPhase.py
index d3fada622..3174cac1a 100644
--- a/pym/_emerge/EbuildPhase.py
+++ b/pym/_emerge/EbuildPhase.py
@@ -11,6 +11,7 @@ from _emerge.BinpkgEnvExtractor import BinpkgEnvExtractor
 from _emerge.MiscFunctionsProcess import MiscFunctionsProcess
 from _emerge.EbuildProcess import EbuildProcess
 from _emerge.CompositeTask import CompositeTask
+from _emerge.PackagePhase import PackagePhase
 from portage.package.ebuild.prepare_build_dirs import (_prepare_workdir,
 		_prepare_fake_distdir, _prepare_fake_filesdir)
 from portage.util import writemsg
@@ -169,6 +170,12 @@ class EbuildPhase(CompositeTask):
 		return logfile
 
 	def _start_ebuild(self):
+		if self.phase == "package":
+			self._start_task(PackagePhase(actionmap=self.actionmap,
+				background=self.background, fd_pipes=self.fd_pipes,
+				logfile=self._get_log_path(), scheduler=self.scheduler,
+				settings=self.settings), self._ebuild_exit)
+			return
 
 		if self.phase == "unpack":
 			alist = self.settings.configdict["pkg"].get("A", "").split()
diff --git a/pym/_emerge/PackagePhase.py b/pym/_emerge/PackagePhase.py
new file mode 100644
index 000000000..083745059
--- /dev/null
+++ b/pym/_emerge/PackagePhase.py
@@ -0,0 +1,80 @@
+# Copyright 2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+import io
+
+from _emerge.CompositeTask import CompositeTask
+from _emerge.EbuildProcess import EbuildProcess
+from _emerge.SpawnProcess import SpawnProcess
+
+import portage
+from portage import os
+from portage import _encodings
+from portage import _unicode_encode
+
+
+class PackagePhase(CompositeTask):
+	"""
+	Invokes the package phase and handles PKG_INSTALL_MASK.
+	"""
+
+	__slots__ = ("actionmap", "fd_pipes", "logfile", "settings",
+		"_pkg_install_mask", "_proot")
+
+	_shell_binary = portage.const.BASH_BINARY
+
+	def _start(self):
+		try:
+			with io.open(_unicode_encode(
+				os.path.join(self.settings["PORTAGE_BUILDDIR"],
+				"build-info", "PKG_INSTALL_MASK"),
+				encoding=_encodings['fs'], errors='strict'),
+				mode='r', encoding=_encodings['repo.content'],
+				errors='replace') as f:
+				self._pkg_install_mask = f.read().split()
+		except OSError:
+			self._pkg_install_mask = None
+		if self._pkg_install_mask:
+			self._proot = os.path.join(self.settings['T'], 'packaging')
+			self._start_task(SpawnProcess(
+				args=[self._shell_binary, '-e', '-c', ('rm -rf {PROOT}; '
+				'cp -pPR $(cp --help | grep -q -- "^[[:space:]]*-l," && echo -l)'
+				' "${{D}}" {PROOT}').format(PROOT=portage._shell_quote(self._proot))],
+				background=self.background, env=self.settings.environ(),
+				scheduler=self.scheduler, logfile=self.logfile),
+				self._copy_proot_exit)
+		else:
+			self._proot = self.settings['D']
+			self._start_package_phase()
+
+	def _copy_proot_exit(self, proc):
+		if self._default_exit(proc) != os.EX_OK:
+			self.wait()
+		else:
+			self._start_package_phase()
+
+	def _start_package_phase(self):
+		ebuild_process = EbuildProcess(actionmap=self.actionmap,
+			background=self.background, fd_pipes=self.fd_pipes,
+			logfile=self.logfile, phase="package",
+			scheduler=self.scheduler, settings=self.settings)
+
+		if self._pkg_install_mask:
+			d_orig = self.settings["D"]
+			try:
+				self.settings["D"] = self._proot
+				self._start_task(ebuild_process, self._pkg_install_mask_cleanup)
+			finally:
+				self.settings["D"] = d_orig
+		else:
+			self._start_task(ebuild_process, self._default_final_exit)
+
+	def _pkg_install_mask_cleanup(self, proc):
+		if self._default_exit(proc) != os.EX_OK:
+			self.wait()
+		else:
+			self._start_task(SpawnProcess(
+				args=['rm', '-rf', self._proot],
+				background=self.background, env=self.settings.environ(),
+				scheduler=self.scheduler, logfile=self.logfile),
+				self._default_final_exit)
-- 
2.13.6



^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [gentoo-portage-dev] [PATCH 3/4] {,PKG_}INSTALL_MASK: python implementation
  2018-03-27  9:13 [gentoo-portage-dev] [PATCH 0/4] {,PKG_}INSTALL_MASK support for exclusions (bug 651214) Zac Medico
  2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 1/4] {,PKG_}INSTALL_MASK: record value in vardb Zac Medico
  2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 2/4] EbuildPhase: add PackagePhase class for PKG_INSTALL_MASK Zac Medico
@ 2018-03-27  9:13 ` Zac Medico
  2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 4/4] {,PKG_}INSTALL_MASK: Support exclusions (bug 651214) Zac Medico
  3 siblings, 0 replies; 5+ messages in thread
From: Zac Medico @ 2018-03-27  9:13 UTC (permalink / raw
  To: gentoo-portage-dev; +Cc: Michał Górny, Zac Medico

The InstallMask.match code comes from the dblink _is_install_masked
method from portage-mgorny:

https://github.com/mgorny/portage/commit/f5ac3af3216d209618a2f232b4bf720bc8b520ad
---
 bin/misc-functions.sh            |  73 --------------------------
 pym/_emerge/PackagePhase.py      |  13 ++++-
 pym/portage/dbapi/vartree.py     |  24 ++++++---
 pym/portage/util/install_mask.py | 109 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 139 insertions(+), 80 deletions(-)
 create mode 100644 pym/portage/util/install_mask.py

diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index 4fe6ead17..ea2557724 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -317,72 +317,6 @@ postinst_qa_check() {
 	done < <(printf "%s\0" "${qa_checks[@]}" | LC_ALL=C sort -u -z)
 }
 
-install_mask() {
-	local root="$1"
-	shift
-	local install_mask="$*"
-
-	# We think of $install_mask as a space-separated list of
-	# globs. We don't want globbing in the "for" loop; that is, we
-	# want to keep the asterisks in the indivual entries.
-	local shopts=$-
-	set -o noglob
-	local no_inst
-	for no_inst in ${install_mask}; do
-		# Here, $no_inst is a single "entry" potentially
-		# containing a glob. From now on, we *do* want to
-		# expand it.
-		set +o noglob
-
-		# The standard case where $no_inst is something that
-		# the shell could expand on its own.
-		if [[ -e "${root}"/${no_inst} || -L "${root}"/${no_inst} ||
-			"${root}"/${no_inst} != $(echo "${root}"/${no_inst}) ]] ; then
-			__quiet_mode || einfo "Removing ${no_inst}"
-			rm -Rf "${root}"/${no_inst} >&/dev/null
-		fi
-
-		# We also want to allow the user to specify a "bare
-		# glob." For example, $no_inst="*.a" should prevent
-		# ALL files ending in ".a" from being installed,
-		# regardless of their location/depth. We achieve this
-		# by passing the pattern to `find`.
-		find "${root}" \( -path "${no_inst}" -or -name "${no_inst}" \) \
-			-print0 2> /dev/null \
-		| LC_ALL=C sort -z \
-		| while read -r -d ''; do
-			__quiet_mode || einfo "Removing /${REPLY#${root}}"
-			rm -Rf "${REPLY}" >&/dev/null
-		done
-
-	done
-	# set everything back the way we found it
-	set +o noglob
-	set -${shopts}
-}
-
-preinst_mask() {
-	if [ -z "${D}" ]; then
-		 eerror "${FUNCNAME}: D is unset"
-		 return 1
-	fi
-
-	if ! ___eapi_has_prefix_variables; then
-		local ED=${D}
-	fi
-
-	# Make sure $PWD is not ${D} so that we don't leave gmon.out files
-	# in there in case any tools were built with -pg in CFLAGS.
-	cd "${T}"
-
-	install_mask "${ED}" "${INSTALL_MASK}"
-
-	# remove share dir if unnessesary
-	if has nodoc $FEATURES || has noman $FEATURES || has noinfo $FEATURES; then
-		rmdir "${ED%/}/usr/share" &> /dev/null
-	fi
-}
-
 preinst_sfperms() {
 	if [ -z "${D}" ]; then
 		 eerror "${FUNCNAME}: D is unset"
@@ -506,13 +440,6 @@ __dyn_package() {
 	# in there in case any tools were built with -pg in CFLAGS.
 	cd "${T}" || die
 
-	if [[ -n ${PKG_INSTALL_MASK} ]] ; then
-		# The caller makes ${D} refer to a temporary copy in this
-		# case, so that this does not mask files from the normal
-		# install image.
-		install_mask "${D%/}${EPREFIX}/" "${PKG_INSTALL_MASK}"
-	fi
-
 	local tar_options=""
 	[[ $PORTAGE_VERBOSE = 1 ]] && tar_options+=" -v"
 	has xattr ${FEATURES} && [[ $(tar --help 2> /dev/null) == *--xattrs* ]] && tar_options+=" --xattrs"
diff --git a/pym/_emerge/PackagePhase.py b/pym/_emerge/PackagePhase.py
index 083745059..35137532a 100644
--- a/pym/_emerge/PackagePhase.py
+++ b/pym/_emerge/PackagePhase.py
@@ -11,6 +11,8 @@ import portage
 from portage import os
 from portage import _encodings
 from portage import _unicode_encode
+from portage.util._async.AsyncFunction import AsyncFunction
+from portage.util.install_mask import install_mask_dir, InstallMask
 
 
 class PackagePhase(CompositeTask):
@@ -31,7 +33,7 @@ class PackagePhase(CompositeTask):
 				encoding=_encodings['fs'], errors='strict'),
 				mode='r', encoding=_encodings['repo.content'],
 				errors='replace') as f:
-				self._pkg_install_mask = f.read().split()
+				self._pkg_install_mask = InstallMask(f.read())
 		except OSError:
 			self._pkg_install_mask = None
 		if self._pkg_install_mask:
@@ -51,6 +53,15 @@ class PackagePhase(CompositeTask):
 		if self._default_exit(proc) != os.EX_OK:
 			self.wait()
 		else:
+			self._start_task(AsyncFunction(
+				target=install_mask_dir,
+				args=(self._proot, self._pkg_install_mask)),
+				self._pkg_install_mask_exit)
+
+	def _pkg_install_mask_exit(self, proc):
+		if self._default_exit(proc) != os.EX_OK:
+			self.wait()
+		else:
 			self._start_package_phase()
 
 	def _start_package_phase(self):
diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index bed76d80f..bf7bd8c2a 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -32,6 +32,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
 		'grabdict,normalize_path,new_protect_filename',
 	'portage.util.digraph:digraph',
 	'portage.util.env_update:env_update',
+	'portage.util.install_mask:install_mask_dir,InstallMask',
 	'portage.util.listdir:dircache,listdir',
 	'portage.util.movefile:movefile',
 	'portage.util.path:first_existing,iter_parents',
@@ -3841,15 +3842,26 @@ class dblink(object):
 					max_dblnk = dblnk
 			self._installed_instance = max_dblnk
 
-		# Apply INSTALL_MASK before collision-protect, since it may
+		# Update INSTALL_MASK before collision-protect, since it may
 		# be useful to avoid collisions in some scenarios.
 		# We cannot detect if this is needed or not here as INSTALL_MASK can be
 		# modified by bashrc files.
-		phase = MiscFunctionsProcess(background=False,
-			commands=["preinst_mask"], phase="preinst",
-			scheduler=self._scheduler, settings=self.settings)
-		phase.start()
-		phase.wait()
+		try:
+			with io.open(_unicode_encode(os.path.join(inforoot, "INSTALL_MASK"),
+				encoding=_encodings['fs'], errors='strict'),
+				mode='r', encoding=_encodings['repo.content'],
+				errors='replace') as f:
+				install_mask = InstallMask(f.read())
+		except OSError:
+			install_mask = None
+
+		if install_mask:
+			install_mask_dir(self.settings["ED"], install_mask)
+			if any(x in self.settings.features for x in ('nodoc', 'noman', 'noinfo')):
+				try:
+					os.rmdir(os.path.join(self.settings["ED"], 'usr', 'share'))
+				except OSError:
+					pass
 
 		# We check for unicode encoding issues after src_install. However,
 		# the check must be repeated here for binary packages (it's
diff --git a/pym/portage/util/install_mask.py b/pym/portage/util/install_mask.py
new file mode 100644
index 000000000..64fe0b21a
--- /dev/null
+++ b/pym/portage/util/install_mask.py
@@ -0,0 +1,109 @@
+# Copyright 2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+__all__ = ['install_mask_dir', 'InstallMask']
+
+import errno
+import fnmatch
+import os
+
+from portage.exception import (
+	OperationNotPermitted, PermissionDenied, FileNotFound)
+from portage.util import normalize_path
+
+
+class InstallMask(object):
+	def __init__(self, install_mask):
+		"""
+		@param install_mask: INSTALL_MASK value
+		@type install_mask: str
+		"""
+		self._install_mask = install_mask.split()
+
+	def match(self, path):
+		"""
+		@param path: file path relative to ${ED}
+		@type path: str
+		@rtype: bool
+		@return: True if path matches INSTALL_MASK, False otherwise
+		"""
+		ret = False
+		for pattern in self._install_mask:
+			# absolute path pattern
+			if pattern.startswith('/'):
+				# match either exact path or one of parent dirs
+				# the latter is done via matching pattern/*
+				if (fnmatch.fnmatch(path, pattern[1:])
+						or fnmatch.fnmatch(path, pattern[1:] + '/*')):
+					ret = True
+					break
+			# filename
+			else:
+				if fnmatch.fnmatch(os.path.basename(path), pattern):
+					ret = True
+					break
+		return ret
+
+
+_exc_map = {
+	errno.ENOENT: FileNotFound,
+	errno.EPERM: OperationNotPermitted,
+	errno.EACCES: PermissionDenied,
+}
+
+
+def _raise_exc(e):
+	"""
+	Wrap OSError with portage.exception wrapper exceptions, with
+	__cause__ chaining when python supports it.
+
+	@param e: os exception
+	@type e: OSError
+	@raise PortageException: portage.exception wrapper exception
+	"""
+	wrapper_cls = _exc_map.get(e.errno)
+	if wrapper_cls is None:
+		raise
+	wrapper = wrapper_cls(str(e))
+	wrapper.__cause__ = e
+	raise wrapper
+
+
+def install_mask_dir(base_dir, install_mask, onerror=None):
+	"""
+	Remove files and directories matched by INSTALL_MASK.
+
+	@param base_dir: directory path corresponding to ${ED}
+	@type base_dir: str
+	@param install_mask: INSTALL_MASK configuration
+	@type install_mask: InstallMask
+	"""
+	onerror = onerror or _raise_exc
+	base_dir = normalize_path(base_dir)
+	base_dir_len = len(base_dir) + 1
+	dir_stack = []
+
+	# Remove masked files.
+	for parent, dirs, files in os.walk(base_dir, onerror=onerror):
+		dir_stack.append(parent)
+		for fname in files:
+			abs_path = os.path.join(parent, fname)
+			relative_path = abs_path[base_dir_len:]
+			if install_mask.match(relative_path):
+				try:
+					os.unlink(abs_path)
+				except OSError as e:
+					onerror(e)
+
+	# Remove masked dirs (unless non-empty due to exclusions).
+	while True:
+		try:
+			dir_path = dir_stack.pop()
+		except IndexError:
+			break
+
+		if install_mask.match(dir_path[base_dir_len:]):
+			try:
+				os.rmdir(dir_path)
+			except OSError:
+				pass
-- 
2.13.6



^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [gentoo-portage-dev] [PATCH 4/4] {,PKG_}INSTALL_MASK: Support exclusions (bug 651214)
  2018-03-27  9:13 [gentoo-portage-dev] [PATCH 0/4] {,PKG_}INSTALL_MASK support for exclusions (bug 651214) Zac Medico
                   ` (2 preceding siblings ...)
  2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 3/4] {,PKG_}INSTALL_MASK: python implementation Zac Medico
@ 2018-03-27  9:13 ` Zac Medico
  3 siblings, 0 replies; 5+ messages in thread
From: Zac Medico @ 2018-03-27  9:13 UTC (permalink / raw
  To: gentoo-portage-dev; +Cc: Michał Górny

From: Michał Górny <mgorny@gentoo.org>

Allow INSTALL_MASK patterns to start with '-' to indicate that
a specific match is to be excluded from being masked. In this case,
the last matching pattern determines whether the file is actually
filtered out or kept.

Bug: https://bugs.gentoo.org/651214
---
 pym/portage/util/install_mask.py | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/pym/portage/util/install_mask.py b/pym/portage/util/install_mask.py
index 64fe0b21a..92989d71d 100644
--- a/pym/portage/util/install_mask.py
+++ b/pym/portage/util/install_mask.py
@@ -29,19 +29,21 @@ class InstallMask(object):
 		"""
 		ret = False
 		for pattern in self._install_mask:
+			# if pattern starts with -, possibly exclude this path
+			is_inclusive = not pattern.startswith('-')
+			if not is_inclusive:
+				pattern = pattern[1:]
 			# absolute path pattern
 			if pattern.startswith('/'):
 				# match either exact path or one of parent dirs
 				# the latter is done via matching pattern/*
 				if (fnmatch.fnmatch(path, pattern[1:])
 						or fnmatch.fnmatch(path, pattern[1:] + '/*')):
-					ret = True
-					break
+					ret = is_inclusive
 			# filename
 			else:
 				if fnmatch.fnmatch(os.path.basename(path), pattern):
-					ret = True
-					break
+					ret = is_inclusive
 		return ret
 
 
-- 
2.13.6



^ permalink raw reply related	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2018-03-27  9:20 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-03-27  9:13 [gentoo-portage-dev] [PATCH 0/4] {,PKG_}INSTALL_MASK support for exclusions (bug 651214) Zac Medico
2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 1/4] {,PKG_}INSTALL_MASK: record value in vardb Zac Medico
2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 2/4] EbuildPhase: add PackagePhase class for PKG_INSTALL_MASK Zac Medico
2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 3/4] {,PKG_}INSTALL_MASK: python implementation Zac Medico
2018-03-27  9:13 ` [gentoo-portage-dev] [PATCH 4/4] {,PKG_}INSTALL_MASK: Support exclusions (bug 651214) Zac Medico

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox