[gentoo-portage-dev] [PATCH] dblink.treewalk: lock for parallel-install with blockers (bug 576888)

[gentoo-portage-dev] [PATCH] dblink.treewalk: lock for parallel-install with blockers (bug 576888)

[Zac Medico]

For parallel-install, lock when interacting with blocked packages, in
order to account for blocked packages being removed or replaced
concurrently.

X-Gentoo-Bug:     576888
X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=576888
---
 pym/portage/dbapi/vartree.py | 45 ++++++++++++++++++++++++++++++++++----------
 1 file changed, 35 insertions(+), 10 deletions(-)

diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index e7effca..6d4514c 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -3932,9 +3932,25 @@ class dblink(object):
 		blockers = self._blockers
 		if blockers is None:
 			blockers = []
-		collisions, dirs_ro, symlink_collisions, plib_collisions = \
-			self._collision_protect(srcroot, destroot,
-			others_in_slot + blockers, filelist, linklist)
+
+		# For parallel-install, acquire a lock to interact
+		# with blocked packages.
+		with_lock = bool(blockers) and "parallel-install" in self.settings.features
+
+		if with_lock:
+			self.lockdb()
+			self.vartree.dbapi._fs_lock()
+			# Filter out any blocked packages that may have
+			# been uninstalled concurrently.
+			blockers = [blocker for blocker in blockers if blocker.exists()]
+		try:
+			collisions, dirs_ro, symlink_collisions, plib_collisions = \
+				self._collision_protect(srcroot, destroot,
+				others_in_slot + blockers, filelist, linklist)
+		finally:
+			if with_lock:
+				self.vartree.dbapi._fs_unlock()
+				self.unlockdb()
 
 		# Check for read-only filesystems.
 		ro_checker = get_ro_checker()
@@ -4290,13 +4306,22 @@ class dblink(object):
 		self._clear_contents_cache()
 		contents = self.getcontents()
 		destroot_len = len(destroot) - 1
-		self.lockdb()
-		try:
-			for blocker in blockers:
-				self.vartree.dbapi.removeFromContents(blocker, iter(contents),
-					relative_paths=False)
-		finally:
-			self.unlockdb()
+
+		if blockers:
+			self.lockdb()
+			self.vartree.dbapi._fs_lock()
+			try:
+				for blocker in blockers:
+					# Blocked packages may have been removed
+					# or replaced concurrently, so check if
+					# they still exist, and clear contents cache.
+					if blocker.exists():
+						blocker._clear_contents_cache()
+						self.vartree.dbapi.removeFromContents(blocker,
+							iter(contents), relative_paths=False)
+			finally:
+				self.vartree.dbapi._fs_unlock()
+				self.unlockdb()
 
 		plib_registry = self.vartree.dbapi._plib_registry
 		if plib_registry:
-- 
2.4.10

[gentoo-portage-dev] Re: [PATCH] dblink.treewalk: lock for parallel-install with blockers (bug 576888)

[Zac Medico]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

On Wed, Mar 9, 2016 at 1:57 PM, Zac Medico <zmedico@gentoo.org> wrote:

> For parallel-install, lock when interacting with blocked packages, in
> order to account for blocked packages being removed or replaced
> concurrently.
>
> X-Gentoo-Bug:     576888
> X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=576888
> ---
>  pym/portage/dbapi/vartree.py | 45
> ++++++++++++++++++++++++++++++++++----------
>  1 file changed, 35 insertions(+), 10 deletions(-)
>

 I need to think some more about how this some more. I'm afraid that
concurrent uninstall of one of the blocked packages could unmerge files
that were just installed by the current package.

[-- Attachment #2: Type: text/html, Size: 1095 bytes --]