From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <gentoo-commits+bounces-1484563-garchives=archives.gentoo.org@lists.gentoo.org>
Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by finch.gentoo.org (Postfix) with ESMTPS id 9725215800F
for <garchives@archives.gentoo.org>; Tue, 7 Feb 2023 08:10:15 +0000 (UTC)
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id AE0EAE0788;
Tue, 7 Feb 2023 08:10:14 +0000 (UTC)
Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
(No client certificate requested)
by pigeon.gentoo.org (Postfix) with ESMTPS id 90599E0788
for <gentoo-commits@lists.gentoo.org>; Tue, 7 Feb 2023 08:10:14 +0000 (UTC)
Received: from oystercatcher.gentoo.org (oystercatcher.gentoo.org [148.251.78.52])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
(No client certificate requested)
by smtp.gentoo.org (Postfix) with ESMTPS id 6A474340CF6
for <gentoo-commits@lists.gentoo.org>; Tue, 7 Feb 2023 08:10:13 +0000 (UTC)
Received: from localhost.localdomain (localhost [IPv6:::1])
by oystercatcher.gentoo.org (Postfix) with ESMTP id 952DD7FB
for <gentoo-commits@lists.gentoo.org>; Tue, 7 Feb 2023 08:10:11 +0000 (UTC)
From: "Fabian Groffen" <grobian@gentoo.org>
To: gentoo-commits@lists.gentoo.org
Content-Transfer-Encoding: 8bit
Content-type: text/plain; charset=UTF-8
Reply-To: gentoo-dev@lists.gentoo.org, "Fabian Groffen" <grobian@gentoo.org>
Message-ID: <1675755133.1102e2e474a24cffa6cab5118ab757b89ebf5be7.grobian@gentoo>
Subject: [gentoo-commits] proj/portage-utils:master commit in: /
X-VCS-Repository: proj/portage-utils
X-VCS-Files: qmerge.c
X-VCS-Directories: /
X-VCS-Committer: grobian
X-VCS-Committer-Name: Fabian Groffen
X-VCS-Revision: 1102e2e474a24cffa6cab5118ab757b89ebf5be7
X-VCS-Branch: master
Date: Tue, 7 Feb 2023 08:10:11 +0000 (UTC)
Precedence: bulk
List-Post: <mailto:gentoo-commits@lists.gentoo.org>
List-Help: <mailto:gentoo-commits+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-commits+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-commits+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-commits.gentoo.org>
X-BeenThere: gentoo-commits@lists.gentoo.org
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-Archives-Salt: 8e6c4b82-a67b-41a6-af0d-5561dbfe972b
X-Archives-Hash: 4c629672679ce88e8db64525b8924d10
commit: 1102e2e474a24cffa6cab5118ab757b89ebf5be7
Author: Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Feb 7 07:32:13 2023 +0000
Commit: Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue Feb 7 07:32:13 2023 +0000
URL: https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=1102e2e4
qmerge: add some braindump notes about how things could be
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>
qmerge.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 94 insertions(+)
diff --git a/qmerge.c b/qmerge.c
index 18fdbb5..246a733 100644
--- a/qmerge.c
+++ b/qmerge.c
@@ -47,6 +47,100 @@
--noscript don't execute pkg_{pre,post}{inst,rm} (if any)
*/
+/* How things should work, ideally. This is not how it currently is
+ * implemented at all.
+ *
+ * invocation: qmerge ... pkg/a pkg/b pkg/c
+ * initial mergeset: pkg/a pkg/b pkg/c
+ * resolving:
+ * - for pkg, get dependencies
+ * * apply masks
+ * * extract depend, rdepend, bdepend, etc.
+ * * filter flags in use (libq/dep)
+ * * add found pkgs to mergeset if not present in resolvedset
+ * - while mergeset contains pkgs
+ * * resolve pkg (see above)
+ * * move from mergeset to resolvedset
+ * here (if all is well) mergeset is empty, and resolvedset contains the
+ * pkgs to be installed. If for instance pkg/c depends on pkg/b it
+ * won't occur double, for it is a set.
+ *
+ * Technically, because we deal with binpkgs, we don't have
+ * dependencies, but there can be pre/post scripts that actually use
+ * depended on pkgs, and if we would invoke ebuild(5) to compile instead
+ * of unpack, we would respect the order too, so the resolvedset here
+ * needs to be ordered into a list.
+ * While functionally one can re-evaluate the dependencies here,
+ * implementation wise, it probably is easier to build the final merge
+ * order list while resolving. This can also add the additional
+ * metadata of whether a pkg was requested (cmdline) or pulled in a dep,
+ * and for which package. That information can be leveraged to draw a
+ * tree (visuals) but also to determine possible parallel installation
+ * paths.
+ *
+ * For example, original invocation could lead to a resolved merge list
+ * of:
+ * M pkg/a
+ * m pkg/d
+ * R pkg/c
+ * M pkg/b
+ *
+ * After this, the pkgs need to be fetched and the pre phases need to be
+ * run. In interactive mode, a question probably needs to be inserted
+ * after the printing of the resolved merge list. Then if all checks
+ * out, the unpack and merge to live fs + vdb updates can be performed.
+ * Ideally the unpack (or compile via ebuild(5)) phase is separated from
+ * the final merge to live fs. The latter always has to be serial, but
+ * the former can run in parallel based on dependencies seen. For
+ * our example, pkg/d and pkg/b can be unpacked in parallel, merged in
+ * which order finished first, then pkg/a and pkg/c can commence. So
+ * the start set is pkg/d and pkg/b, and each unlocks pkg/a and pkg/c
+ * respectively, which are not constrained, other than the final merge
+ * logic.
+ *
+ * Errors
+ * There are multiple kinds of errors in multiple stages. Whether they
+ * are fatal depends on a number of factors.
+ * - resolution error
+ * Failing to resolve an atom basically makes the tree that that pkg
+ * is part of unmergable; that is, it needs to be discarded from the
+ * workset. In interactive mode after resolving we could ask if the
+ * user wants to continue (if there's anything else we *can* do),
+ * non-interactive we could decide to just go ahead with whatever we
+ * was possible, unless we add a flag that stops at resolution errors.
+ * - USE-dep resolution error
+ * The state of USE-flags must be respected, and can cause problems,
+ * in particular cyclic dependencies. Solution to those is to disable
+ * USE-flags temporary and re-merge without. For now, these errors
+ * are not resolved, but should be detected and treated as resolution
+ * errors above.
+ * - fetch error
+ * Either because fetching the binpkg or the source files fails. This
+ * discards the atom and its tree. It may be possible in this case to
+ * try and re-resolve using an older version of the pkg. But since
+ * this kind of error is pretty much in the foundation of the work, it
+ * seems more logical to exclude the tree the package belongs too,
+ * because at this point parallel execution happens, it makes no sense
+ * any more to ask the user to abort.
+ * - unpack or merge error
+ * Under these errors are the failures in the various pkg checks (run
+ * phases) and for source-based installs obviously compilation
+ * failures. These discard an entire tree, and like fetch errors,
+ * we don't have a clear opportunity anymore to ask whether or not to
+ * continue.
+ * - live fs + vdb error
+ * This should be rare, but most probably filesystem related errors,
+ * such as running out of diskspace or lacking certain permissions.
+ * Corrupting the VDB hopefully doesn't happen, but it is possible to
+ * encounter problems there as well. Like fetch and unpack errors, we
+ * should try to continue with whatever we can, but will not roll-back
+ * already merged packages. So a failure here, should result in
+ * dropping all children from the failed pkg.
+ *
+ * After merging qlop -Ev should show whatever was merged successfully,
+ * so qmerge should show what failed to merge (in what stage).
+ */
+
/* #define BUSYBOX "/bin/busybox" */
#define BUSYBOX ""