[gentoo-commits] proj/portage:master commit in: doc/config/, lib/portage/_sets/, cnf/sets/

[gentoo-commits] proj/portage:master commit in: doc/config/, lib/portage/_sets/, cnf/sets/

[Zac Medico]

commit:     6c70596d6d6382160490bb16ef559ccfdd982fae
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Sat Jan 16 02:47:09 2021 +0000
Commit:     Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Jan 17 10:35:03 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=6c70596d

Add @changed-subslot package set

This set is the upgradable packages for which the highest visible
version has a different subslot than the currently installed version.

The primary purpose of this feature is for use in catalyst builds. We
update the "seed" stage3 before using it to build a new stage1.

Updating the entire stage is expensive and unnecessary (since we're
going to build the latest packages in stage1 and then rebuild everything
in stage3).

What we definitely do need to update in the original stage3 however, is
any package that would trigger a subslot rebuild.

For example: gcc links with libmpfr.so from dev-libs/mpfr. mpfr's SONAME
changes from libmpfr.so.4 (SLOT="0/4") to libmpfr.so.6 (SLOT="0/6"). If
the seed stage's dev-libs/mpfr is not updated before emerging gcc, gcc
will link with libmpfr.so.4, but the latest version of dev-libs/mpfr
will be built and libmpfr.so.6 included into the stage1. Since the old
libmpfr.so.4 is not included in the stage1, gcc will not work, breaking
subsequent stage builds.

Our current options to update the seed are too large a hammer (e.g.,
"--update --deep --newuse @world" or "--update --deep --newuse
--complete-graph --rebuild-if-new-ver gcc") and spend too much time
updating seed stages for no gain beyond updating only packages for whom
the subslot has changed.

With this set, catalyst will likely use

	emerge @changed-subslot --ignore-built-slot-operator-deps y

to update the seed stage.

Thank you to Zac Medico for showing me how to do this.

Bug: https://bugs.gentoo.org/739004
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>

 cnf/sets/portage.conf      |  5 +++++
 doc/config/sets.docbook    |  8 ++++++++
 lib/portage/_sets/dbapi.py | 39 +++++++++++++++++++++++++++++++++++++--
 3 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/cnf/sets/portage.conf b/cnf/sets/portage.conf
index 22f0fa3a5..5651a9c53 100644
--- a/cnf/sets/portage.conf
+++ b/cnf/sets/portage.conf
@@ -84,6 +84,11 @@ exclude-files = /usr/bin/Xorg
 [rebuilt-binaries]
 class = portage.sets.dbapi.RebuiltBinaries
 
+# Installed packages for which the subslot of the highest visible ebuild
+# version is different than the currently installed version.
+[changed-subslot]
+class = portage.sets.dbapi.SubslotChangedSet
+
 # Installed packages for which the highest visible ebuild
 # version is lower than the currently installed version.
 [downgrade]

diff --git a/doc/config/sets.docbook b/doc/config/sets.docbook
index 0fe50f252..eba98f468 100644
--- a/doc/config/sets.docbook
+++ b/doc/config/sets.docbook
@@ -574,6 +574,14 @@
 		to check.</para></listitem>
 		</itemizedlist>
 		</sect2>
+		<sect2 id='config-set-classes-SubslotChangedSet'>
+		<title>portage.sets.dbapi.SubslotChangedSet</title>
+		<para>
+		Package set which contains all packages for which the subslot of the highest
+		visible ebuild is different than the currently installed version.
+		This class doesn't support any extra options.
+		</para>
+		</sect2>
 		<sect2 id='config-set-classes-DowngradeSet'>
 		<title>portage.sets.dbapi.DowngradeSet</title>
 		<para>

diff --git a/lib/portage/_sets/dbapi.py b/lib/portage/_sets/dbapi.py
index 52367c4a6..7a20553b5 100644
--- a/lib/portage/_sets/dbapi.py
+++ b/lib/portage/_sets/dbapi.py
@@ -1,4 +1,4 @@
-# Copyright 2007-2020 Gentoo Authors
+# Copyright 2007-2021 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 import glob
@@ -15,7 +15,7 @@ from portage._sets import SetConfigError, get_boolean
 import portage
 
 __all__ = ["CategorySet", "ChangedDepsSet", "DowngradeSet",
-	"EverythingSet", "OwnerSet", "VariableSet"]
+	"EverythingSet", "OwnerSet", "SubslotChangedSet", "VariableSet"]
 
 class EverythingSet(PackageSet):
 	_operations = ["merge"]
@@ -167,6 +167,41 @@ class VariableSet(EverythingSet):
 
 	singleBuilder = classmethod(singleBuilder)
 
+class SubslotChangedSet(PackageSet):
+
+	_operations = ["merge", "unmerge"]
+
+	description = "Package set which contains all packages " + \
+		"for which the subslot of the highest visible ebuild is " + \
+		"different than the currently installed version."
+
+	def __init__(self, portdb=None, vardb=None):
+		super(SubslotChangedSet, self).__init__()
+		self._portdb = portdb
+		self._vardb = vardb
+
+	def load(self):
+		atoms = []
+		xmatch = self._portdb.xmatch
+		xmatch_level = "bestmatch-visible"
+		cp_list = self._vardb.cp_list
+		for cp in self._vardb.cp_all():
+			for pkg in cp_list(cp):
+				slot_atom = "%s:%s" % (pkg.cp, pkg.slot)
+				ebuild = xmatch(xmatch_level, slot_atom)
+				if not ebuild:
+					continue
+				if pkg.sub_slot != ebuild.sub_slot:
+					atoms.append(slot_atom)
+
+		self._setAtoms(atoms)
+
+	def singleBuilder(cls, options, settings, trees):
+		return cls(portdb=trees["porttree"].dbapi,
+			vardb=trees["vartree"].dbapi)
+
+	singleBuilder = classmethod(singleBuilder)
+
 class DowngradeSet(PackageSet):
 
 	_operations = ["merge", "unmerge"]