[gentoo-commits] proj/portage-utils:master commit in: libq/

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     93b42c880a13128663c6ddf2ee1a98b901ede6fa
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 13:25:15 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 13:25:15 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=93b42c88

libq/tree: avoid invalid read in tree_foreach_packages

eat_file doesn't return how much data was written, only produced buffer
size, so need to calculate length there

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index e4c4eb7..eb5f324 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1277,11 +1277,11 @@ tree_close_pkg(tree_pkg_ctx *pkg_ctx)
 static int
 tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 {
-	char *p = ctx->pkgs;
+	char *p;
 	char *q;
 	char *c;
 	char pkgname[_Q_PATH_MAX];
-	size_t len = ctx->pkgslen;
+	size_t len;
 	int ret = 0;
 	depend_atom *query = ctx->query_atom;
 
@@ -1307,6 +1307,9 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 		close(fd);
 	}
 
+	p = ctx->pkgs;
+	len = strlen(ctx->pkgs);  /* sucks, need eat_file change */
+
 	memset(&meta, 0, sizeof(meta));
 
 	do {

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     191fc5a7cf9cec475ab74eb42525502a27d9e586
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Jun 29 10:41:48 2024 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Jun 29 10:41:48 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=191fc5a7

libq/Makefile: fix out-of-srcdir build

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/Makefile.am | 3 ++-
 libq/Makefile.in | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/libq/Makefile.am b/libq/Makefile.am
index e65bb47..428d012 100644
--- a/libq/Makefile.am
+++ b/libq/Makefile.am
@@ -33,4 +33,5 @@ libq_a_SOURCES = $(QFILES)
 libq_a_CPPFLAGS = \
 	$(OPENMP_CFLAGS) \
 	-I$(top_builddir)/autotools/gnulib \
-	-I$(top_srcdir)/autotools/gnulib
+	-I$(top_srcdir)/autotools/gnulib \
+	-I$(top_srcdir)

diff --git a/libq/Makefile.in b/libq/Makefile.in
index fdbfb65..6f45c93 100644
--- a/libq/Makefile.in
+++ b/libq/Makefile.in
@@ -1711,7 +1711,8 @@ libq_a_SOURCES = $(QFILES)
 libq_a_CPPFLAGS = \
 	$(OPENMP_CFLAGS) \
 	-I$(top_builddir)/autotools/gnulib \
-	-I$(top_srcdir)/autotools/gnulib
+	-I$(top_srcdir)/autotools/gnulib \
+	-I$(top_srcdir)
 
 all: all-am
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     9594126239e0d21b88c3b8c535b6635b4a8b8892
Author:     Boris Staletic <boris.staletic <AT> protonmail <DOT> com>
AuthorDate: Fri Mar 29 17:30:05 2024 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Apr  8 19:26:59 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=95941262

libq/xarray: Handle NULL arrays in xarraysort()

Some invocations of `q` may try to call `xarraysort(NULL, 0)`.
One example is `qlop -a foo` where `foo` was never unmerged.

Instead of requiring every call of `xarraysort()` to take care of `NULL`
arguments, `xarraysort()` now exits early if `arr->eles == NULL`.

Closes: https://github.com/gentoo/portage-utils/pull/28
Signed-off-by: Boris Staletic <boris.staletic <AT> protonmail.com>
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xarray.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libq/xarray.c b/libq/xarray.c
index 49b478b..3251a12 100644
--- a/libq/xarray.c
+++ b/libq/xarray.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2003-2019 Gentoo Foundation
+ * Copyright 2003-2024 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2003-2007 Ned Ludd        - <solar@gentoo.org>
@@ -46,7 +46,8 @@ void *xarraypush(array_t *arr, const void *ele, size_t ele_len)
 
 void xarraysort(array_t *arr, int (*compar)(const void *, const void *))
 {
-	qsort(arr->eles, arr->num, sizeof(void *), compar);
+	if (arr->num > 1)
+		qsort(arr->eles, arr->num, sizeof(void *), compar);
 }
 
 void xarraydelete_ptr(array_t *arr, size_t elem)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     13b0eecccaeae428c5fcd58b9c7edf9854e154a3
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Feb  1 08:20:42 2024 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Feb  1 08:20:42 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=13b0eecc

libq/tree: handle hypothetical fail in tree_pkg_meta_get_int

When we cannot read all bytes from a file, return an empty string, not
partial garbage.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index 15d8267..4678634 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1233,6 +1233,12 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 					p[--s.st_size] = '\0';
 				m->storage->pos += s.st_size + 1;
 			}
+			else
+			{
+				/* hmmm, couldn't read the whole file?!? */
+				p[0] = '\0';
+				m->storage->pos++;
+			}
 			close(fd);
 		}
 	} else {

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     b4ace2f3443e6746a54eb14c7f50aa719540181c
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Feb  1 08:19:01 2024 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Feb  1 08:19:01 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=b4ace2f3

libq/contents: fix invalid access problem pointed out by valgrind

len represents the entire string length, but we start scanning after the
line identifier, so substract that size from len, such that we don't
start scanning after the end of the input string.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/contents.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/libq/contents.c b/libq/contents.c
index feb1c0b..6ec4491 100644
--- a/libq/contents.c
+++ b/libq/contents.c
@@ -34,6 +34,9 @@ contents_parse_line_len(char *line, size_t len)
 		len--;
 	}
 
+	if (len <= 4)  /* minimal: "dir /" */
+		return NULL;
+
 	memset(&e, 0x00, sizeof(e));
 	e._data = line;
 
@@ -47,6 +50,7 @@ contents_parse_line_len(char *line, size_t len)
 		return NULL;
 
 	e.name = e._data + 4;
+	len   -= 4;
 
 	switch (e.type) {
 		/* dir /bin */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     17e518250d3cabfec9ecc417275a42858b590297
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Jan 31 20:39:24 2024 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Jan 31 20:39:24 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=17e51825

libq/contents: add variant specifying buffer length

This seems necessary for PR #21, but keep the original code structure
largely in-tact.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/contents.c | 53 +++++++++++++++++++++++++++++++++++------------------
 libq/contents.h |  5 +++--
 2 files changed, 38 insertions(+), 20 deletions(-)

diff --git a/libq/contents.c b/libq/contents.c
index 7f4351d..feb1c0b 100644
--- a/libq/contents.c
+++ b/libq/contents.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2020 Gentoo Foundation
+ * Copyright 2005-2024 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
@@ -19,18 +19,20 @@
  * Parse a line of CONTENTS file and provide access to the individual fields
  */
 contents_entry *
-contents_parse_line(char *line)
+contents_parse_line_len(char *line, size_t len)
 {
 	static contents_entry e;
 	char *p;
 
-	if (line == NULL || *line == '\0' || *line == '\n')
+	if (len == 0 || line == NULL || *line == '\0' || *line == '\n')
 		return NULL;
 
 	/* chop trailing newline */
-	p = &line[strlen(line) - 1];
-	if (*p == '\n')
+	p = &line[len - 1];
+	if (*p == '\n') {
 		*p = '\0';
+		len--;
+	}
 
 	memset(&e, 0x00, sizeof(e));
 	e._data = line;
@@ -53,23 +55,38 @@ contents_parse_line(char *line)
 
 		/* obj /bin/bash 62ed51c8b23866777552643ec57614b0 1120707577 */
 		case CONTENTS_OBJ:
-			if ((e.mtime_str = strrchr(e.name, ' ')) == NULL)
-				return NULL;
-			*e.mtime_str++ = '\0';
-			if ((e.digest = strrchr(e.name, ' ')) == NULL)
-				return NULL;
-			*e.digest++ = '\0';
+			for (p = &e.name[len - 1]; p >= e.name; p--) {
+				if (*p == ' ') {
+					if (e.mtime_str == NULL)
+						e.mtime_str = p + 1;
+					else if (e.digest == NULL)
+						e.digest = p + 1;
+					*p = '\0';
+
+					if (e.digest != NULL)
+						break;
+				}
+			}
 			break;
 
 		/* sym /bin/sh -> bash 1120707577 */
 		case CONTENTS_SYM:
-			if ((e.mtime_str = strrchr(e.name, ' ')) == NULL)
-				return NULL;
-			*e.mtime_str++ = '\0';
-			if ((e.sym_target = strstr(e.name, " -> ")) == NULL)
-				return NULL;
-			*e.sym_target = '\0';
-			e.sym_target += 4;
+			for (p = &e.name[len - 1]; p >= e.name; p--) {
+				if (*p == ' ') {
+					if (e.mtime_str == NULL) {
+						e.mtime_str = p + 1;
+					} else if (e.sym_target == NULL) {
+						if (strncmp(p, " -> ", sizeof(" -> ") - 1) == 0)
+							e.sym_target = p + sizeof(" -> ") - 1;
+						else
+							continue;
+					}
+					*p = '\0';
+
+					if (e.sym_target != NULL)
+						break;
+				}
+			}
 			break;
 	}
 

diff --git a/libq/contents.h b/libq/contents.h
index c766827..a0a5a63 100644
--- a/libq/contents.h
+++ b/libq/contents.h
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2024 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
@@ -24,6 +24,7 @@ typedef struct {
 	long mtime;
 } contents_entry;
 
-contents_entry *contents_parse_line(char *line);
+contents_entry *contents_parse_line_len(char *line, size_t len);
+#define contents_parse_line(L) contents_parse_line_len(L, strlen(L))
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     014a97249e20e77f3e0ba5b9def194682ef8ecbd
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Jan 31 19:30:24 2024 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Jan 31 19:30:24 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=014a9724

libq/hash.h: update copyright for previous commit

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/hash.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/hash.h b/libq/hash.h
index ffbd2ef..bc72e52 100644
--- a/libq/hash.h
+++ b/libq/hash.h
@@ -1,5 +1,5 @@
 /*
- * Copyright 2018-2020 Gentoo Foundation
+ * Copyright 2018-2024 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2018-     Fabian Groffen  - <grobian@gentoo.org>

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     bc4321f30bb95ab1c2112f045a4cde811045ed59
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Jan 31 19:24:41 2024 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Jan 31 19:24:41 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=bc4321f3

libq/hash: add hash_string function

Alternative to the implementation in PR #21, so as to reuse the same
hashing code.

We could add the interface to compute multiple hashes from the same
string when that's actually necessary.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/hash.c | 137 +++++++++++++++++++++++++++++++++++++++++++++++++-----------
 libq/hash.h |   1 +
 2 files changed, 114 insertions(+), 24 deletions(-)

diff --git a/libq/hash.c b/libq/hash.c
index 9b36bb9..f3a440f 100644
--- a/libq/hash.c
+++ b/libq/hash.c
@@ -98,29 +98,54 @@ hash_hex(char *out, const unsigned char *buf, const int length)
 	}
 }
 
-/**
- * Computes the hashes for file fname and writes the hex-representation
- * for those hashes into the address space pointed to by the return
- * pointers for these hashes.  The caller should ensure enough space is
- * available.  Only those hashes which are in the global hashes variable
- * are computed, the address space pointed to for non-used hashes are
- * left untouched, e.g. they can be NULL.  The number of bytes read from
- * the file pointed to by fname is returned in the flen argument.
- */
-int
-hash_multiple_file_fd(
-		int fd,
-		char *md5,
-		char *sha1,
-		char *sha256,
-		char *sha512,
-		char *blak2b,
+/* len func(dest,destlen,cbctx) */
+typedef size_t (*read_cb)(char *,size_t,void *);
+
+static size_t read_stdio(char *dest, size_t destlen, void *ctx)
+{
+	FILE *io = ctx;
+
+	return fread(dest, 1, destlen, io);
+}
+
+struct bufctx {
+	const char *buf;
+	size_t      buflen;
+};
+
+static size_t read_buffer(char *dest, size_t destlen, void *ctx)
+{
+	struct bufctx *membuf = ctx;
+	size_t         readlen;
+
+	readlen = destlen;
+	if (readlen > membuf->buflen)
+		readlen = membuf->buflen;
+
+	memcpy(dest, membuf->buf, readlen);
+
+	/* update buffer to the remainder */
+	membuf->buf    += readlen;
+	membuf->buflen -= readlen;
+
+	return readlen;
+}
+
+static int
+hash_multiple_internal(
+		read_cb rcb,
+		void   *ctx,
+		char   *md5,
+		char   *sha1,
+		char   *sha256,
+		char   *sha512,
+		char   *blak2b,
 		size_t *flen,
-		int hashes)
+		int     hashes)
 {
-	FILE             *f;
-	char              data[8192];
 	size_t            len;
+	char              data[8192];
+
 	struct md5_ctx    m5;
 	struct sha1_ctx   s1;
 	struct sha256_ctx s256;
@@ -132,8 +157,6 @@ hash_multiple_file_fd(
 #endif
 
 	*flen = 0;
-	if ((f = fdopen(fd, "r")) == NULL)
-		return -1;
 
 	md5_init_ctx(&m5);
 	sha1_init_ctx(&s1);
@@ -143,7 +166,7 @@ hash_multiple_file_fd(
 	blake2b_init(&bl2b, BLAKE2B_OUTBYTES);
 #endif
 
-	while ((len = fread(data, 1, sizeof(data), f)) > 0) {
+	while ((len = rcb(data, sizeof(data), ctx)) > 0) {
 		*flen += len;
 #pragma omp parallel sections
 		{
@@ -176,7 +199,6 @@ hash_multiple_file_fd(
 #endif
 		}
 	}
-	fclose(f);
 
 #pragma omp parallel sections
 	{
@@ -227,6 +249,41 @@ hash_multiple_file_fd(
 	return 0;
 }
 
+/**
+ * Computes the hashes for file fname and writes the hex-representation
+ * for those hashes into the address space pointed to by the return
+ * pointers for these hashes.  The caller should ensure enough space is
+ * available.  Only those hashes which are in the global hashes variable
+ * are computed, the address space pointed to for non-used hashes are
+ * left untouched, e.g. they can be NULL.  The number of bytes read from
+ * the file pointed to by fname is returned in the flen argument.
+ */
+int
+hash_multiple_file_fd(
+		int fd,
+		char *md5,
+		char *sha1,
+		char *sha256,
+		char *sha512,
+		char *blak2b,
+		size_t *flen,
+		int hashes)
+{
+	FILE *f;
+	int   ret;
+
+	if ((f = fdopen(fd, "r")) == NULL)
+		return -1;
+
+	ret = hash_multiple_internal(read_stdio, f,
+								 md5, sha1, sha256, sha512, blak2b,
+								 flen, hashes);
+
+	fclose(f);
+
+	return ret;
+}
+
 int
 hash_multiple_file_at_cb(
 		int pfd,
@@ -285,3 +342,35 @@ hash_file_at_cb(int pfd, const char *fname, int hash, hash_cb_t cb)
 
 	return _hash_file_buf;
 }
+
+char *
+hash_string(const char *buf, ssize_t buflen, int hash)
+{
+	struct bufctx membuf;
+	size_t        dummy;
+
+	if (buflen < 0)
+		buflen = (ssize_t)strlen(buf);
+
+	membuf.buf    =  buf;
+	membuf.buflen = (size_t)buflen;
+
+	switch (hash) {
+		case HASH_MD5:
+		case HASH_SHA1:
+		case HASH_SHA256:
+		case HASH_SHA512:
+		case HASH_BLAKE2B:
+			if (hash_multiple_internal(read_buffer, &membuf,
+									   _hash_file_buf, _hash_file_buf,
+									   _hash_file_buf, _hash_file_buf,
+									   _hash_file_buf,
+									   &dummy, hash) != 0)
+				return NULL;
+			break;
+		default:
+			return NULL;
+	}
+
+	return _hash_file_buf;
+}

diff --git a/libq/hash.h b/libq/hash.h
index fb4ab5f..ffbd2ef 100644
--- a/libq/hash.h
+++ b/libq/hash.h
@@ -44,5 +44,6 @@ int hash_multiple_file_at_cb(
 char *hash_file_at_cb(int pfd, const char *filename, int hash_algo, hash_cb_t cb);
 #define hash_file(f, h) hash_file_at_cb(AT_FDCWD, f, h, NULL)
 #define hash_file_at(fd, f, h) hash_file_at_cb(fd, f, h, NULL)
+char *hash_string(const char *buf, ssize_t buflen, int hash);
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e05b6baf296397bc2a10dad728f2840ab242b833
Author:     Pavel Kalugin <pavel <AT> pavelthebest <DOT> me>
AuthorDate: Tue Dec 12 19:32:35 2023 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Jan 27 13:27:30 2024 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e05b6baf

libq/atom: fix atom comparison bug

qlop SEGFAULTed when predict on a package without category was called

Closes: https://github.com/gentoo/portage-utils/pull/24
Signed-off-by: Pavel Kalugin <pavel <AT> pavelthebest.me>
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/libq/atom.c b/libq/atom.c
index 31299f1..b1a150a 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -1252,7 +1252,15 @@ atom_compar_cb(const void *l, const void *r)
 		default:
 		{
 			int ret;
-			ret = strcmp(al->CATEGORY, ar->CATEGORY);
+			if (!al->CATEGORY && !ar->CATEGORY) {
+				ret = 0;
+			} else if (!al->CATEGORY) {
+				ret = -1;
+			} else if (!ar->CATEGORY) {
+				ret = 1;
+			} else {
+				ret = strcmp(al->CATEGORY, ar->CATEGORY);
+			}
 			if (ret == 0)
 				ret = strcasecmp(al->PN, ar->PN);
 			return ret;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e322a78cbfd6d51aefe26425dff1cb99c3d307bc
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Apr 21 19:09:05 2023 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Apr 21 19:09:05 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e322a78c

libq/tree: add commetns on file checks (research for bug #898194)

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index f308c8d..1922b7d 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1109,8 +1109,17 @@ tree_pkg_read(tree_pkg_ctx *pkg_ctx)
 
 	if (ctx->cachetype == CACHE_METADATA_MD5) {
 		ret = tree_read_file_md5(pkg_ctx);
+		/* md5-cache, is sort of documented in egencache man-page
+		 * key-points are that an md5 is provided for the ebuild itself,
+		 * and if it includes eclasses, the md5s for each eclass.  These
+		 * are available as _md5_ and _eclasses_ keys.  The latter uses
+		 * tab-separation of form <eclass-name>\t<md5>\t... */
 	} else if (ctx->cachetype == CACHE_METADATA_PMS) {
 		ret = tree_read_file_pms(pkg_ctx);
+		/* PMS implies to do an mtime and existence check (the cache may
+		 * contain extra stuff) but since this form of metadata in fact
+		 * is extinct, because these checks are insufficient and
+		 * impossible on e.g. a git-based tree. */
 	} else if (ctx->cachetype == CACHE_EBUILD) {
 		ret = tree_read_file_ebuild(pkg_ctx);
 	} else if (ctx->cachetype == CACHE_BINPKGS) {

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     933b7cdbd8e15c6b120aec9cb5bec3ec36a27485
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Jan 30 14:08:37 2023 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Jan 30 14:08:37 2023 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=933b7cdb

libq/tree: add support IDEPEND

Bug: https://bugs.gentoo.org/892533
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 6 +++++-
 libq/tree.h | 5 +++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 2a457ce..76190ed 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2022 Gentoo Foundation
+ * Copyright 2005-2023 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
@@ -824,6 +824,7 @@ tree_read_file_md5(tree_pkg_ctx *pkg_ctx)
 		assign_var(DEFINED_PHASES);
 		assign_var(REQUIRED_USE);
 		assign_var(BDEPEND);
+		assign_var(IDEPEND);
 		assign_var(EPREFIX);
 		assign_var(_eclasses_);
 		assign_var(_md5_);
@@ -901,6 +902,7 @@ tree_read_file_ebuild(tree_pkg_ctx *pkg_ctx)
 			match_key(EAPI);
 			match_key(REQUIRED_USE);
 			match_key(BDEPEND);
+			match_key(IDEPEND);
 #undef match_key
 		}
 
@@ -1004,6 +1006,7 @@ tree_read_file_binpkg_xpak_cb(
 	match_path(DEFINED_PHASES);
 	match_path(REQUIRED_USE);
 	match_path(BDEPEND);
+	match_path(IDEPEND);
 	match_path(CONTENTS);
 	match_path(USE);
 	match_path(EPREFIX);
@@ -1513,6 +1516,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 		match_key2(REPO, repository);
 		match_key(SIZE);
 		match_key(BDEPEND);
+		match_key(IDEPEND);
 		match_key(PATH);
 		match_key2(BUILD_ID, BUILDID);
 #undef match_key

diff --git a/libq/tree.h b/libq/tree.h
index 2f2c81f..efafe73 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2022 Gentoo Foundation
+ * Copyright 2005-2023 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  */
 
@@ -96,8 +96,9 @@ struct tree_pkg_meta {
 	char *Q_PDEPEND;
 	char *Q_PROVIDE;       /* line 14 */
 	char *Q_EAPI;
-	char *Q_PROPERTIES;
+	char *Q_PROPERTIES;    /* last line from metadata */
 	char *Q_BDEPEND;
+	char *Q_IDEPEND;
 	/* binpkgs/vdb */
 	char *Q_DEFINED_PHASES;
 	char *Q_REQUIRED_USE;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     8ca9d1625448544f72aa0d45a1cbbdfa8dcfb22e
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu May 26 09:27:11 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu May 26 09:27:11 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=8ca9d162

libq/dep: add dep_resolve_tree function

allow resolving a dep-specification to atoms found in a tree, e.g. see
what would get selected

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/dep.c | 77 +++++++++++++++++++++++++++++++++++++++-----------------------
 libq/dep.h | 12 ++++++----
 2 files changed, 56 insertions(+), 33 deletions(-)

diff --git a/libq/dep.c b/libq/dep.c
index 99629e7..4138a1c 100644
--- a/libq/dep.c
+++ b/libq/dep.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2022 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
@@ -20,6 +20,7 @@
 #include "atom.h"
 #include "dep.h"
 #include "set.h"
+#include "tree.h"
 #include "xarray.h"
 #include "xasprintf.h"
 
@@ -62,8 +63,6 @@ static void
 _dep_burn_node(dep_node *node)
 {
 	assert(node);
-	if (node->info_on_heap)
-		free(node->info);
 	if (node->atom)
 		atom_implode(node->atom);
 	free(node);
@@ -237,7 +236,6 @@ dep_print_tree(
 {
 	size_t s;
 	int indent = 4;  /* Gentoo 4-wide indent standard */
-	depend_atom *d = NULL;
 	bool singlechild = false;
 	bool nonewline = false;
 
@@ -260,33 +258,31 @@ dep_print_tree(
 	if (root->type == DEP_OR)
 		fprintf(fp, "|| (");
 	if (root->info) {
-		if (hlatoms != NULL && array_cnt(hlatoms) > 0 &&
-				root->type == DEP_NORM)
-		{
-			size_t i;
-			depend_atom *m;
-			char *oslot;
-
-			d = root->atom;
-			d->pfx_op = d->sfx_op = ATOM_OP_NONE;
-
-			array_for_each(hlatoms, i, m) {
-				oslot = d->SLOT;
-				if (m->SLOT == NULL)
-					d->SLOT = NULL;
-
-				if (atom_compare(m, d) == EQUAL) {
-					m = NULL;
-					break;
+		if (root->type == DEP_NORM) {
+			bool dohl = false;
+
+			if (hlatoms != NULL && array_cnt(hlatoms) > 0)
+			{
+				size_t       i;
+				depend_atom *m;
+
+				array_for_each(hlatoms, i, m) {
+					/* make m query, such that any specifics (SLOT,
+					 * pfx/sfx) from the depstring are ignored while
+					 * highlighting */
+					if (atom_compare(root->atom, m) == EQUAL) {
+						dohl = true;
+						break;
+					}
 				}
-				d->SLOT = oslot;
 			}
 
-			if (m == NULL) { /* match found */
-				fprintf(fp, "%s%s%s", hlcolor, root->info, NORM);
-			} else {
-				fprintf(fp, "%s", root->info);
-			}
+			fprintf(fp, "%s%s%s",
+					dohl ? hlcolor : "",
+					atom_to_string(root->atom),
+					dohl ? NORM : "");
+			if (root->atom_resolved && verbose > 0)
+				fprintf(fp, "  # %s", root->info);
 		} else {
 			fprintf(fp, "%s", root->info);
 		}
@@ -355,6 +351,31 @@ dep_prune_use(dep_node *root, set *use)
 		dep_prune_use(root->children, use);
 }
 
+void
+dep_resolve_tree(dep_node *root, tree_ctx *t)
+{
+	if (root->type != DEP_NULL) {
+		if (root->type == DEP_NORM && root->atom) {
+			depend_atom    *d = root->atom;
+			tree_match_ctx *r = tree_match_atom(t, d,
+											 	TREE_MATCH_DEFAULT |
+											 	TREE_MATCH_LATEST);
+			if (r != NULL) {
+				atom_implode(d);
+				root->atom = atom_clone(r->atom);
+				root->atom_resolved = 1;
+				tree_match_close(r);
+			}
+		}
+
+		if (root->children)
+			dep_resolve_tree(root->children, t);
+	}
+
+	if (root->neighbor)
+		dep_resolve_tree(root->neighbor, t);
+}
+
 void
 dep_flatten_tree(const dep_node *root, array_t *out)
 {

diff --git a/libq/dep.h b/libq/dep.h
index 1055d29..68d5c75 100644
--- a/libq/dep.h
+++ b/libq/dep.h
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2022 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  */
 
@@ -9,6 +9,7 @@
 #include "atom.h"
 #include "colors.h"
 #include "set.h"
+#include "tree.h"
 #include "xarray.h"
 
 typedef enum {
@@ -28,10 +29,10 @@ static const char * const _dep_names[] = {
 };
 
 struct _dep_node {
-	dep_type type;
-	char *info;
-	char info_on_heap;
-	depend_atom *atom;
+	dep_type          type;
+	char             *info;
+	char              atom_resolved:1;
+	depend_atom      *atom;
 	struct _dep_node *parent;
 	struct _dep_node *neighbor;
 	struct _dep_node *children;
@@ -47,6 +48,7 @@ typedef struct _dep_node dep_node;
 
 dep_node *dep_grow_tree(const char *depend);
 void dep_print_tree(FILE *fp, const dep_node *root, size_t space, array_t *m, const char *c, int verbose);
+void dep_resolve_tree(dep_node *root, tree_ctx *t);
 void dep_burn_tree(dep_node *root);
 void dep_prune_use(dep_node *root, set *use);
 void dep_flatten_tree(const dep_node *root, array_t *out);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     84804b1dfcf6e5f3b7a7cca2212c8b2d5a7e4f4b
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu May 26 09:30:28 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu May 26 09:30:28 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=84804b1d

libq/dep: allow resolving multiple times (e.g. for overlays)

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/dep.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/dep.c b/libq/dep.c
index 4138a1c..389e506 100644
--- a/libq/dep.c
+++ b/libq/dep.c
@@ -355,7 +355,7 @@ void
 dep_resolve_tree(dep_node *root, tree_ctx *t)
 {
 	if (root->type != DEP_NULL) {
-		if (root->type == DEP_NORM && root->atom) {
+		if (root->type == DEP_NORM && root->atom && !root->atom_resolved) {
 			depend_atom    *d = root->atom;
 			tree_match_ctx *r = tree_match_atom(t, d,
 											 	TREE_MATCH_DEFAULT |

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e1a3cb832e35df4656e64b3385823d1ffd8ab848
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri May 20 17:14:54 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri May 20 17:14:54 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e1a3cb83

libq/atom: make atom_to_string_r produce BUILD_ID when set

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libq/atom.c b/libq/atom.c
index 3d9d31f..50e9520 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -1015,6 +1015,8 @@ atom_to_string_r(char *buf, size_t buflen, depend_atom *a)
 		off += snprintf(buf + off, buflen - off, "-%s", a->PV);
 	if (a->PR_int > 0)
 		off += snprintf(buf + off, buflen - off, "-r%d", a->PR_int);
+	if (a->BUILDID > 0)
+		off += snprintf(buf + off, buflen - off, "~%u", a->BUILDID);
 	off += snprintf(buf + off, buflen - off, "%s", atom_op_str[a->sfx_op]);
 	if (a->SLOT != NULL || a->slotdep != ATOM_SD_NONE)
 		off += snprintf(buf + off, buflen - off, ":%s%s%s%s",

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     b868d22a6c731449ccfec344508458cf9f7a5abf
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri May 20 17:02:56 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri May 20 17:02:56 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=b868d22a

libq/tree: support FEATURES=binpkg-multi-instance Packages file

Based on the work of genbtc in GitHub PR #16

Add BUILD_ID to atom when parsing and use PATH to get the appropriate
location.

Closes: https://github.com/gentoo/portage-utils/pull/16
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 23 ++++++++++++++++++-----
 libq/tree.h |  4 +++-
 2 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index d71ee74..2a457ce 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1440,6 +1440,8 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 					}
 					cat->pkg_ctxs = (tree_pkg_ctx **)atom;  /* for name */
 				}
+				if (meta.Q_BUILDID != NULL)
+					atom->BUILDID = atoi(meta.Q_BUILDID);
 				pkgnamelen = snprintf(pkgname, sizeof(pkgname),
 						"%s.tbz2", atom->PF);
 				pkgname[pkgnamelen - (sizeof(".tbz2") - 1)] = '\0';
@@ -1510,6 +1512,9 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 		match_key(PDEPEND);
 		match_key2(REPO, repository);
 		match_key(SIZE);
+		match_key(BDEPEND);
+		match_key(PATH);
+		match_key2(BUILD_ID, BUILDID);
 #undef match_key
 #undef match_key2
 		}
@@ -1812,11 +1817,19 @@ tree_match_atom(tree_ctx *ctx, const depend_atom *query, int flags)
 			n = xzalloc(sizeof(tree_match_ctx)); \
 			n->atom = atom; \
 			n->pkg = pkg_ctx; \
-			snprintf(n->path, sizeof(n->path), "%s/%s/%s%s", \
-					(char *)C->ctx->path, C->name, pkg_ctx->name, \
-					C->ctx->cachetype == CACHE_EBUILD ? ".ebuild" : \
-					C->ctx->cachetype == CACHE_BINPKGS ? ".tbz2" : \
-					C->ctx->cachetype == CACHE_PACKAGES ? ".tbz2" : ""); \
+			if (C->ctx->cachetype == CACHE_PACKAGES && \
+				pkg_ctx->meta->Q_PATH != NULL) \
+			{ \
+				/* binpkg-multi-instance has a PATH ready for us */ \
+				snprintf(n->path, sizeof(n->path), "%s/%s", \
+						 (char *)C->ctx->path, pkg_ctx->meta->Q_PATH); \
+			} else { \
+				snprintf(n->path, sizeof(n->path), "%s/%s/%s%s", \
+						 (char *)C->ctx->path, C->name, pkg_ctx->name, \
+						 C->ctx->cachetype == CACHE_EBUILD   ? ".ebuild" : \
+						 C->ctx->cachetype == CACHE_BINPKGS  ? ".tbz2" : \
+						 C->ctx->cachetype == CACHE_PACKAGES ? ".tbz2" : ""); \
+			} \
 			if (flags & TREE_MATCH_METADATA) \
 				n->meta = tree_pkg_read(pkg_ctx); \
 			if (C->ctx->cachetype == CACHE_BINPKGS || \

diff --git a/libq/tree.h b/libq/tree.h
index 8279281..2f2c81f 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2021 Gentoo Foundation
+ * Copyright 2005-2022 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  */
 
@@ -104,6 +104,8 @@ struct tree_pkg_meta {
 	char *Q_CONTENTS;
 	char *Q_USE;
 	char *Q_EPREFIX;
+	char *Q_PATH;     /* binpkg-multi-instance */
+	char *Q_BUILDID;  /* binpkg-multi-instance */
 	char *Q_repository;
 	char *Q_MD5;
 	char *Q_SHA1;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     566c8b8db5bc0f7ae4636a1d3387ddb6de41692f
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu May 19 08:31:40 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu May 19 08:31:40 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=566c8b8d

libq/atom: allow including BUILDID in atom_format

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/libq/atom.c b/libq/atom.c
index 6f88698..3d9d31f 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -1045,6 +1045,7 @@ atom_to_string_r(char *buf, size_t buflen, depend_atom *a)
  *    - any prefix of these (e.g. CAT, CA, C) will match as well
  *  pfx - the version qualifier if set (e.g. > < = !)
  *  sfx - the version qualifier if set (e.g. *)
+ *  BUILDID - the binpkg-multi-instance id
  */
 char *
 atom_format_r(
@@ -1174,6 +1175,20 @@ atom_format_r(
 								append_buf(buf, buflen, "%s", "]");
 						}
 					}
+				} else if (strncmp("BUILDID", fmt, len) == 0) {
+					if (showit || atom->BUILDID > 0) {
+						/* this is really shitty, '-' is not feasible,
+						 * but used by Portage
+						 * https://archives.gentoo.org/gentoo-portage-dev/message/054f5f1f334b60bdb1b7f80ff4755bd4
+						 * using this we cannot parse what we would
+						 * produce, but look more like the original
+						 * since it's not clear this is necessary at
+						 * all, I decided to avoid any confusion and use
+						 * '~' so we can see this is not a version bit */
+						append_buf(buf, buflen, "%s%s%u%s",
+								   RED, connected ? "~" : "",
+								   atom->BUILDID, NORM);
+					}
 				} else
 					append_buf(buf, buflen, "<BAD:%.*s>", (int)len, fmt);
 				p++;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     be7090b77585c7c4af0d46ca0cde805d370c6cc0
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu May 19 08:15:34 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu May 19 08:15:34 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=be7090b7

libq/atom: make atom_compare consider BUILDID

PR: https://github.com/gentoo/portage-utils/pull/16
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index d1eb9a4..6f88698 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -954,16 +954,24 @@ atom_compare_flg(const depend_atom *data, const depend_atom *query, int flags)
 	 * 6:    return  A < B
 	 * 7:  end if
 	 */
-	/* Make sure the -r# is the same. 3.7 */
+	/* first handle wildcarding cases */
 	if ((sfx_op == ATOM_OP_STAR && query->PR_int == 0) ||
 	    pfx_op == ATOM_OP_PV_EQUAL ||
-		flags & ATOM_COMP_NOREV ||
-	    data->PR_int == query->PR_int)
+		flags & ATOM_COMP_NOREV)
 		return _atom_compare_match(EQUAL, pfx_op);
-	else if (data->PR_int < query->PR_int)
+	/* Make sure the -r# is the same. 3.7 */
+	if (data->PR_int < query->PR_int)
+		return _atom_compare_match(OLDER, pfx_op);
+	else if (data->PR_int > query->PR_int)
+		return _atom_compare_match(NEWER, pfx_op);
+
+	/* binpkg-multi-instance support */
+	if (data->BUILDID < query->BUILDID)
 		return _atom_compare_match(OLDER, pfx_op);
-	else
+	if (data->BUILDID > query->BUILDID)
 		return _atom_compare_match(NEWER, pfx_op);
+
+	return _atom_compare_match(EQUAL, pfx_op);
 }
 
 atom_equality

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     74aaef62c0bfad13cf528497faf4b12cbe3d52a5
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu May 19 07:43:08 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu May 19 07:43:08 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=74aaef62

libq/atom.h: add BUILDID to atom for binpkg-multi-instance support

References: https://github.com/gentoo/portage-utils/pull/16
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.h | 33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)

diff --git a/libq/atom.h b/libq/atom.h
index 8291daf..fcfa0bd 100644
--- a/libq/atom.h
+++ b/libq/atom.h
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2021 Gentoo Foundation
+ * Copyright 2005-2022 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
@@ -68,23 +68,24 @@ typedef struct _atom_usedep {
 } atom_usedep;
 
 typedef struct {
-	atom_blocker blocker;
+	atom_blocker  blocker;
 	atom_operator pfx_op;
 	atom_operator sfx_op;
-	char *CATEGORY;
-	char *PN;
-	char *PV;
-	char *PF;
-	unsigned int PR_int;
-	char letter;
-	atom_suffix *suffixes;
-	char *PVR;
-	char *P;
-	atom_usedep *usedeps;
-	char *SLOT;
-	char *SUBSLOT;
-	atom_slotdep slotdep;
-	char *REPO;
+	char         *CATEGORY;
+	char         *PN;
+	char         *PV;
+	char         *PF;
+	unsigned int  PR_int;
+	char          letter;
+	atom_suffix  *suffixes;
+	char         *PVR;
+	char         *P;
+	atom_usedep  *usedeps;
+	char         *SLOT;
+	char         *SUBSLOT;
+	atom_slotdep  slotdep;
+	char         *REPO;
+	unsigned int  BUILDID;
 } depend_atom;
 
 extern const char * const booga[];

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     4b8b0aa52cac041938a08a310f40b6de16b8c3b0
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 12 17:12:02 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Feb 12 17:12:02 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=4b8b0aa5

libq/move_file: fix Coverity uninitialised use CID 248870

Coverity correctly deduced here, that if we used cached stat, we didn't
use the cache, but a bogus memory struct instead.  Ensure we always use
a populated stat value.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/move_file.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libq/move_file.c b/libq/move_file.c
index b98c8e2..104ca25 100644
--- a/libq/move_file.c
+++ b/libq/move_file.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2021 Gentoo Authors
+ * Copyright 2005-2022 Gentoo Authors
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
@@ -86,8 +86,8 @@ move_file(int rootfd_src, const char *name_src,
 		}
 
 		/* preserve the file times */
-		times[0] = get_stat_atime(&st);
-		times[1] = get_stat_mtime(&st);
+		times[0] = get_stat_atime(stat_src);
+		times[1] = get_stat_mtime(stat_src);
 		futimens(fd_dst, times);
 
 		close(fd_src);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     d314b816ef29bf1948cd9f747a22c2565e88bf76
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 12 17:03:56 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Feb 12 17:03:56 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=d314b816

libq/tree: fix Coverity uninitialised memory use CID 248874

tree_read_file_pms was using buf, which was never initialised after
some changes from the past

remove entire buf, and use the paths from pkg and cat ctxs in the
printfs directly

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 1e0e623..d71ee74 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -676,7 +676,6 @@ tree_read_file_pms(tree_pkg_ctx *pkg_ctx)
 	FILE *f;
 	tree_pkg_meta *ret = NULL;
 	size_t len;
-	char buf[_Q_PATH_MAX];
 
 	if ((f = fdopen(pkg_ctx->fd, "r")) == NULL)
 		goto err;
@@ -695,7 +694,8 @@ tree_read_file_pms(tree_pkg_ctx *pkg_ctx)
 	ret->Q_DEPEND = ptr;
 #define next_line(curr, next) \
 	if ((ptr = strchr(ret->Q_##curr, '\n')) == NULL) { \
-		warn("Invalid cache file for '%s'", buf); \
+		warn("Invalid cache file for '%s/%s'", \
+			 pkg_ctx->cat_ctx->name, pkg_ctx->name); \
 		goto err; \
 	} \
 	ret->Q_##next = ptr+1; \
@@ -718,8 +718,8 @@ tree_read_file_pms(tree_pkg_ctx *pkg_ctx)
 #undef next_line
 	ptr = strchr(ptr+1, '\n');
 	if (ptr == NULL) {
-		warn("Invalid cache file for '%s' - could not find end of cache data",
-				buf);
+		warn("Invalid cache file for '%s/%s' - could not find end of cache data",
+			 pkg_ctx->cat_ctx->name, pkg_ctx->name);
 		goto err;
 	}
 	*ptr = '\0';

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     42a04c468aff667e0dbaf69aef07c9f679176470
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Feb  6 14:51:14 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Feb  6 14:51:14 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=42a04c46

libq/tree: plug leak in tree_match_atom

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libq/tree.c b/libq/tree.c
index 7285cd7..1e0e623 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1778,6 +1778,7 @@ tree_match_atom(tree_ctx *ctx, const depend_atom *query, int flags)
 						sizeof(*cat_ctx->pkg_ctxs), tree_pkg_compar);
 			}
 		}
+		xarrayfree_int(cats);
 	}
 
 	/* activate cache for future lookups, tree_match_atom relies on

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     43ad7423ae06e9bcad672a21131f14e3ce790204
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Feb  6 14:20:35 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Feb  6 14:20:35 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=43ad7423

libq/xsystem: cleanup/reuse same codepath, allow passing vector

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xsystem.c | 96 ++++++++++++++++++++++++++++++++++++----------------------
 libq/xsystem.h |  8 +++--
 2 files changed, 65 insertions(+), 39 deletions(-)

diff --git a/libq/xsystem.c b/libq/xsystem.c
index e2dbc5e..05743ce 100644
--- a/libq/xsystem.c
+++ b/libq/xsystem.c
@@ -17,49 +17,73 @@
 #include "xasprintf.h"
 #include "xsystem.h"
 
-void xsystem(const char *command)
-{
-	if (unlikely(system(command)))
-		errp("system(%s) failed", command);
-}
-
-void xsystembash(const char *command, int cwd)
+void xsystembash(const char *command, const char **argv, int cwd)
 {
 	pid_t p = fork();
 	int status;
 
 	switch (p) {
-	case 0: /* child */
-		if (cwd != AT_FDCWD)
-			if (fchdir(cwd)) {
-				/* fchdir works with O_PATH starting w/linux-3.5 */
-				if (errno == EBADF) {
-					char *path;
-					xasprintf(&path, "/proc/self/fd/%i", cwd);
-					if (chdir(path))
-						errp("chdir(%s) failed", path);
-				} else
-					errp("fchdir(%i) failed", cwd);
+		case 0: /* child */
+			if (cwd != AT_FDCWD) {
+				if (fchdir(cwd)) {
+					/* fchdir works with O_PATH starting w/linux-3.5 */
+					if (errno == EBADF) {
+						char *path;
+						xasprintf(&path, "/proc/self/fd/%i", cwd);
+						if (chdir(path))
+							errp("chdir(%s) failed", path);
+					} else {
+						errp("fchdir(%i) failed", cwd);
+					}
+				}
+			}
+			if (argv == NULL) {
+				execl(CONFIG_EPREFIX "bin/bash", "bash",
+					  "--norc", "--noprofile", "-c", command, (char *)NULL);
+				/* Hrm, still here ?  Maybe no bash ... */
+				_exit(execl("/bin/sh", "sh", "-c", command, (char *)NULL));
+			} else {
+				int          argc = 0;
+				const char  *a;
+				const char **newargv;
+
+				/* count existing args */
+				for (a = argv[0]; a != NULL; a++, argc++)
+					;
+				argc += 1 + 1 + 1 + 1;
+				newargv = xmalloc(sizeof(newargv[0]) * (argc + 1));
+				argc = 0;
+				newargv[argc++] = "bash";
+				newargv[argc++] = "--norc";
+				newargv[argc++] = "--noprofile";
+				newargv[argc++] = "-c";
+				for (a = argv[0]; a != NULL; a++)
+					newargv[argc++] = a;
+				newargv[argc] = NULL;
+
+				execv(CONFIG_EPREFIX "bin/bash", (char *const *)newargv);
+
+				/* Hrm, still here ?  Maybe no bash ... */
+				newargv = &newargv[2];  /* shift, two args less */
+				argc = 0;
+				newargv[argc++] = "sh";
+				_exit(execv("/bin/sh", (char *const *)newargv));
 			}
-		execl(CONFIG_EPREFIX "bin/bash", "bash",
-				"--norc", "--noprofile", "-c", command, (char *)NULL);
-		/* Hrm, still here ?  Maybe no bash ... */
-		_exit(execl("/bin/sh", "sh", "-c", command, (char *)NULL));
 
-	default: /* parent */
-		waitpid(p, &status, 0);
-		if (WIFSIGNALED(status)) {
-			err("phase crashed with signal %i: %s", WTERMSIG(status),
-			    strsignal(WTERMSIG(status)));
-		} else if (WIFEXITED(status)) {
-			if (WEXITSTATUS(status) == 0)
-				return;
-			else
-				err("phase exited %i", WEXITSTATUS(status));
-		}
-		/* fall through */
+		default: /* parent */
+			waitpid(p, &status, 0);
+			if (WIFSIGNALED(status)) {
+				err("phase crashed with signal %i: %s", WTERMSIG(status),
+					strsignal(WTERMSIG(status)));
+			} else if (WIFEXITED(status)) {
+				if (WEXITSTATUS(status) == 0)
+					return;
+				else
+					err("phase exited %i", WEXITSTATUS(status));
+			}
+			/* fall through */
 
-	case -1: /* fucked */
-		errp("xsystembash(%s) failed", command);
+		case -1: /* fucked */
+			errp("xsystembash(%s) failed", command);
 	}
 }

diff --git a/libq/xsystem.h b/libq/xsystem.h
index 833840d..d6c4fa4 100644
--- a/libq/xsystem.h
+++ b/libq/xsystem.h
@@ -1,14 +1,16 @@
 /*
- * Copyright 2010-2019 Gentoo Foundation
+ * Copyright 2010-2022 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2010-2016 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2022      Fabian Groffen  - <grobian@gentoo.org>
  */
 
 #ifndef _XSYSTEM_H
 #define _XSYSTEM_H 1
 
-void xsystem(const char *command);
-void xsystembash(const char *command, int cwd);
+void xsystembash(const char *command, const char **argv, int cwd);
+#define xsystem(C,F)  xsystembash(C, NULL, F)
+#define xsystemv(V,F) xsystembash(NULL, V, F)
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     0541387d790d9e9c0e0033492bdbdb5c6d6d2f59
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Feb  6 13:27:22 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Feb  6 13:27:22 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=0541387d

libq/tree: allocate enough space in tree_clone_meta

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index db0d2d2..7285cd7 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1135,7 +1135,7 @@ tree_clone_meta(tree_pkg_meta *m)
 	len = sizeof(*ret);
 	for (ptr = &m->Q__data; ptr <= &m->Q__last; ptr++)
 		if (*ptr != NULL)
-			len += strlen(*ptr);
+			len += strlen(*ptr) + 1;
 
 	/* malloc and copy */
 	ret = xzalloc(len);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     d0767ea8eab387c65a90445bfd5d8e6e196c30d6
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Feb  6 13:24:54 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Feb  6 13:24:54 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=d0767ea8

libq/tree: ensure we free cache.store (for Packages file)

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index 114541d..db0d2d2 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2021 Gentoo Foundation
+ * Copyright 2005-2022 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
@@ -194,6 +194,8 @@ tree_close(tree_ctx *ctx)
 
 		xarrayfree_int(t);
 	}
+	if (ctx->cache.store != NULL)
+		free(ctx->cache.store);
 
 	closedir(ctx->dir);
 	/* closedir() above does this for us: */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     8468afb97a92adb0c49aaff461bafc7fc0f72992
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Feb  6 12:21:53 2022 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Feb  6 12:21:53 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=8468afb9

libq/xsystem: avoid using obsolete vfork()

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xsystem.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libq/xsystem.c b/libq/xsystem.c
index e11172e..e2dbc5e 100644
--- a/libq/xsystem.c
+++ b/libq/xsystem.c
@@ -1,8 +1,9 @@
 /*
- * Copyright 2010-2019 Gentoo Foundation
+ * Copyright 2010-2022 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2010-2016 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2022-     Fabian Groffen  - <grobian@gentoo.org>
  */
 
 #include "main.h"
@@ -24,7 +25,7 @@ void xsystem(const char *command)
 
 void xsystembash(const char *command, int cwd)
 {
-	pid_t p = vfork();
+	pid_t p = fork();
 	int status;
 
 	switch (p) {

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     cea99ce1b9b6d0c9ebb496dcd2f1e77ab9ab1dcd
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Dec 29 12:17:32 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Dec 29 12:17:32 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=cea99ce1

libq/tree: avoid nasty realloc magic on tree_pkg_meta

doing reallocs possibly (hight probability) invalidates previously
retrieved pointers, which is really nasty to work with, and error prone,
so instead allocate incremental slabs where necessary

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 206 +++++++++++++++++++++++++++++++-----------------------------
 libq/tree.h |  11 +++-
 2 files changed, 116 insertions(+), 101 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 87df175..114541d 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -651,6 +651,21 @@ tree_pkg_vdb_eat(
 	return ret;
 }
 
+#define tree_meta_alloc_storage(M,SIZ) { \
+	struct tree_pkg_meta_ll *blk; \
+	size_t                   newlen; \
+\
+	/* calculate new block size, ensuring it covers whatever we \
+	 * need to write this iteration */ \
+	newlen     = ((((SIZ) + 1) / BUFSIZ) + 1) * BUFSIZ; \
+	blk        = xmalloc(sizeof(*blk) + newlen); \
+	memset(blk, 0, sizeof(*blk)); \
+	blk->next  = M->storage; \
+	blk->ptr   = (char *)blk + sizeof(*blk); \
+	blk->len   = newlen; \
+	M->storage = blk; \
+}
+
 static tree_pkg_meta *
 tree_read_file_pms(tree_pkg_ctx *pkg_ctx)
 {
@@ -668,8 +683,9 @@ tree_read_file_pms(tree_pkg_ctx *pkg_ctx)
 		goto err;
 
 	len = sizeof(*ret) + s.st_size + 1;
-	ret = xzalloc(len);
-	ptr = (char*)ret + sizeof(*ret);
+	ret = xmalloc(len);
+	memset(ret, 0, sizeof(*ret));
+	ptr = (char *)ret + sizeof(*ret);
 	if ((off_t)fread(ptr, 1, s.st_size, f) != s.st_size)
 		goto err;
 	ptr[s.st_size] = '\0';
@@ -954,15 +970,15 @@ static void
 tree_read_file_binpkg_xpak_cb(
 	void *ctx,
 	char *pathname,
-	int pathname_len,
-	int data_offset,
-	int data_len,
+	int   pathname_len,
+	int   data_offset,
+	int   data_len,
 	char *data)
 {
 	tree_pkg_meta *m = (tree_pkg_meta *)ctx;
-	char **key;
-	size_t pos;
-	size_t len;
+	char         **key;
+	size_t         pos;
+	size_t         len;
 
 #define match_path(K) \
 	else if (pathname_len == (sizeof(#K) - 1) && strcmp(pathname, #K) == 0) \
@@ -994,37 +1010,28 @@ tree_read_file_binpkg_xpak_cb(
 		return;
 #undef match_path
 
-	/* hijack unused members */
-	pos = (size_t)m->Q__eclasses_;
-	len = (size_t)m->Q__md5_;
+	/* get current storage block */
+	if (m->storage != NULL) {
+		pos = m->storage->pos;
+		len = m->storage->len;
+	} else {
+		pos = 0;
+		len = 0;
+	}
 
 	/* trim whitespace (mostly trailing newline) */
 	while (isspace((int)data[data_offset + data_len - 1]))
 		data_len--;
 
 	if (len - pos < (size_t)(data_len + 1)) {
-		char *old_data = m->Q__data;
-		len += (((data_len + 1 - (len - pos)) / BUFSIZ) + 1) * BUFSIZ;
-		m->Q__data = xrealloc(m->Q__data, len);
-
-		/* re-position existing keys */
-		if (old_data != NULL && m->Q__data != old_data) {
-			char **newdata = (char **)m;
-			int elems = sizeof(tree_pkg_meta) / sizeof(char *);
-			while (elems-- > 1)  /* skip Q__data itself */
-				if (newdata[elems] != NULL)
-					newdata[elems] = m->Q__data + (newdata[elems] - old_data);
-		}
-
-		/* set after repositioning! */
-		m->Q__md5_ = (char *)len;
-		m->Q__eclasses_ = (char *)pos;
+		tree_meta_alloc_storage(m, data_len + 1);
+		len = m->storage->len;
+		pos = m->storage->pos;
 	}
 
-	*key = m->Q__data + pos;
+	*key = m->storage->ptr + pos;
 	snprintf(*key, len - pos, "%.*s", data_len, data + data_offset);
-	pos += data_len + 1;
-	m->Q__eclasses_ = (char *)pos;
+	m->storage->pos += data_len + 1;
 }
 
 static tree_pkg_meta *
@@ -1042,33 +1049,23 @@ tree_read_file_binpkg(tree_pkg_ctx *pkg_ctx)
 	if (newfd != -1) {
 		size_t fsize;
 		size_t needlen = 40 + 1 + 19 + 1;
-		size_t pos = (size_t)m->Q__eclasses_;
-		size_t len = (size_t)m->Q__md5_;
+		size_t pos = 0;
+		size_t len = 0;
 
-		if (len - pos < needlen) {
-			char *old_data = m->Q__data;
-			len += (((needlen - (len - pos)) / BUFSIZ) + 1) * BUFSIZ;
-			m->Q__data = xrealloc(m->Q__data, len);
-
-			/* re-position existing keys */
-			if (old_data != NULL && m->Q__data != old_data) {
-				char **newdata = (char **)m;
-				int elems = sizeof(tree_pkg_meta) / sizeof(char *);
-				while (elems-- > 1)  /* skip Q__data itself */
-					if (newdata[elems] != NULL)
-						newdata[elems] =
-							m->Q__data + (newdata[elems] - old_data);
-			}
+		if (m->storage != NULL) {
+			pos = m->storage->pos;
+			len = m->storage->len;
+		}
 
-			/* set after repositioning! */
-			m->Q__md5_ = (char *)len;
-			m->Q__eclasses_ = (char *)pos;
+		if (len - pos < needlen) {
+			tree_meta_alloc_storage(m, needlen);
+			len = m->storage->len;
+			pos = m->storage->pos;
 		}
 
-		m->Q_SHA1 = m->Q__data + pos;
+		m->Q_SHA1 = m->storage->ptr + pos;
 		m->Q_SIZE = m->Q_SHA1 + 40 + 1;
-		pos += needlen;
-		m->Q__eclasses_ = (char *)pos;
+		m->storage->pos += needlen;
 
 		lseek(newfd, 0, SEEK_SET);  /* reposition at the whole file */
 		if (hash_multiple_file_fd(newfd, NULL, m->Q_SHA1, NULL, NULL,
@@ -1123,13 +1120,48 @@ tree_pkg_read(tree_pkg_ctx *pkg_ctx)
 	return ret;
 }
 
+static tree_pkg_meta *
+tree_clone_meta(tree_pkg_meta *m)
+{
+	tree_pkg_meta *ret;
+	size_t         pos = 0;
+	size_t         len = 0;
+	char         **ptr;
+	char          *p;
+
+	/* compute necessary space upfront */
+	len = sizeof(*ret);
+	for (ptr = &m->Q__data; ptr <= &m->Q__last; ptr++)
+		if (*ptr != NULL)
+			len += strlen(*ptr);
+
+	/* malloc and copy */
+	ret = xzalloc(len);
+	p = (char *)ret + sizeof(*ret);
+	for (ptr = &m->Q__data; ptr <= &m->Q__last; ptr++, pos++) {
+		if (*ptr == NULL)
+			continue;
+		*(&ret->Q__data + pos) = p;
+		len = strlen(*ptr) + 1;
+		memcpy(p, *ptr, len);
+		p += len;
+	}
+
+	return ret;
+}
+
 static void
 tree_close_meta(tree_pkg_meta *cache)
 {
+	struct tree_pkg_meta_ll *blk;
+
 	if (cache == NULL)
 		errf("Cache is empty !");
-	if (cache->Q__data != NULL)
-		free(cache->Q__data);
+	while (cache->storage != NULL) {
+		blk = cache->storage->next;
+		free(cache->storage);
+		cache->storage = blk;
+	}
 	free(cache);
 }
 
@@ -1165,37 +1197,26 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 				return NULL;
 			}
 
-			/* hijack unused members */
-			pos = (size_t)m->Q__eclasses_;
-			len = (size_t)m->Q__md5_;
+			if (m->storage != NULL) {
+				pos = m->storage->pos;
+				len = m->storage->len;
+			} else {
+				pos = 0;
+				len = 0;
+			}
 
-			/* TODO: this is an exact copy from tree_read_file_binpkg_xpak_cb */
 			if (len - pos < (size_t)(s.st_size + 1)) {
-				p = m->Q__data;
-				len += (((s.st_size + 1 - (len - pos)) / BUFSIZ) + 1) * BUFSIZ;
-				m->Q__data = xrealloc(m->Q__data, len);
-
-				/* re-position existing keys */
-				if (p != NULL && m->Q__data != p) {
-					char **newdata = (char **)m;
-					int elems = sizeof(tree_pkg_meta) / sizeof(char *);
-					while (elems-- > 1)  /* skip Q__data itself */
-						if (newdata[elems] != NULL)
-							newdata[elems] = m->Q__data + (newdata[elems] - p);
-				}
-
-				/* set after repositioning! */
-				m->Q__md5_ = (char *)len;
-				m->Q__eclasses_ = (char *)pos;
+				tree_meta_alloc_storage(m, s.st_size + 1);
+				pos = m->storage->pos;
+				len = m->storage->len;
 			}
 
-			p = *key = m->Q__data + pos;
+			p = *key = m->storage->ptr + pos;
 			if (read(fd, p, s.st_size) == (ssize_t)s.st_size) {
 				p[s.st_size] = '\0';
 				while (s.st_size > 0 && isspace((int)p[s.st_size - 1]))
 					p[--s.st_size] = '\0';
-				pos += s.st_size + 1;
-				m->Q__eclasses_ = (char *)pos;
+				m->storage->pos += s.st_size + 1;
 			}
 			close(fd);
 		}
@@ -1692,29 +1713,14 @@ tree_match_atom_cache_populate_cb(tree_pkg_ctx *ctx, void *priv)
 	pkg->repo = tctx->repo != NULL ? xstrdup(tctx->repo) : NULL;
 	if (meta != NULL) {
 		pkg->fd = -2;  /* don't try to read, we fill it in here */
-		pkg->meta = xmalloc(sizeof(*pkg->meta));
-		memcpy(pkg->meta, meta, sizeof(*pkg->meta));
-		if (ctx->cat_ctx->ctx->cachetype == CACHE_PACKAGES) {
-			pkg->meta->Q__data = NULL;  /* avoid free here (just to be sure) */
-		} else {  /* CACHE_BINPKG */
-			char **newdata;
-			int    elems;
-			size_t datasize = (size_t)meta->Q__eclasses_;
-
-			pkg->meta->Q__data = xmalloc(sizeof(char) * datasize);
-			memcpy(pkg->meta->Q__data, meta->Q__data, datasize);
-
-			/* re-position keys */
-			newdata = (char **)pkg->meta;
-			elems   = sizeof(tree_pkg_meta) / sizeof(char *);
-			while (elems-- > 1)  /* skip Q__data itself */
-				if (newdata[elems] != NULL)
-					newdata[elems] = pkg->meta->Q__data +
-						(newdata[elems] - meta->Q__data);
-
-			/* drop garbage used for Q__data admin in original case */
-			pkg->meta->Q__eclasses_ = NULL;
-			pkg->meta->Q__md5_      = NULL;
+		if (tctx->cachetype == CACHE_PACKAGES) {
+			/* need to copy, source is based on temp space in foreach */
+			pkg->meta = tree_clone_meta(meta);
+		} else {
+			/* BINPKG case, this one is read/allocated separately from
+			 * xpak archive, so can just take it over */
+			pkg->meta = meta;
+			ctx->meta = NULL;  /* avoid double free */
 		}
 	} else {
 		pkg->meta = NULL;

diff --git a/libq/tree.h b/libq/tree.h
index 052715b..8279281 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -80,7 +80,7 @@ struct tree_pkg_ctx {
 
 /* Ebuild data */
 struct tree_pkg_meta {
-	char *Q__data;
+#define Q__data Q_DEPEND  /* ptr to first member of the struct */
 	char *Q_DEPEND;        /* line 1 */
 	char *Q_RDEPEND;
 	char *Q_SLOT;
@@ -111,6 +111,15 @@ struct tree_pkg_meta {
 	/* These are MD5-Cache only */
 	char *Q__eclasses_;
 	char *Q__md5_;
+#define Q__last Q__md5_  /* ptr to last data member in struct */
+
+	/* for memory allocations backing the pointers above */
+	struct tree_pkg_meta_ll {
+		char                    *ptr;
+		size_t                   len;
+		size_t                   pos;
+		struct tree_pkg_meta_ll *next;
+	}    *storage;
 };
 
 /* Metadata.xml */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     19df31a23a3a5ab08abde6cea7f4b8bfd44fd07c
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Dec 26 13:48:12 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Dec 26 13:48:12 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=19df31a2

libq/scandirat: add filter_self_parent func

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/scandirat.c | 18 +++++++++++++++---
 libq/scandirat.h |  9 +++++++--
 2 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/libq/scandirat.c b/libq/scandirat.c
index f28ba0d..ec4b691 100644
--- a/libq/scandirat.c
+++ b/libq/scandirat.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2021 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
@@ -87,9 +87,21 @@ scandir_free(struct dirent **de, int cnt)
 }
 
 int
-filter_hidden(const struct dirent *dentry)
+filter_hidden(const struct dirent *de)
 {
-	if (dentry->d_name[0] == '.')
+	if (de->d_name[0] == '.')
 		return 0;
 	return 1;
 }
+
+int
+filter_self_parent(const struct dirent *de)
+{
+	if (de->d_name[0] == '.' &&
+		(de->d_name[1] == '\0' ||
+		 (de->d_name[1] == '.' &&
+		  de->d_name[2] == '\0')))
+		return 0;
+
+	return 1;
+}

diff --git a/libq/scandirat.h b/libq/scandirat.h
index 950cbb1..1ac2b50 100644
--- a/libq/scandirat.h
+++ b/libq/scandirat.h
@@ -1,6 +1,10 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2021 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
+ *
+ * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
+ * Copyright 2005-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2021-     Fabian Groffen  - <grobian@gentoo.org>
  */
 
 #ifndef _SCANDIRAT_H
@@ -19,6 +23,7 @@ int scandirat(
 #endif
 
 void scandir_free(struct dirent **de, int cnt);
-int filter_hidden(const struct dirent *dentry);
+int filter_hidden(const struct dirent *de);
+int filter_self_parent(const struct dirent *de);
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     226ee978a882ad95515fb289027646cdbb230515
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Dec 26 13:50:38 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Dec 26 13:50:38 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=226ee978

buildsys: regenerate

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/Makefile.in | 44 ++++++++++++++++++++++++++++----------------
 1 file changed, 28 insertions(+), 16 deletions(-)

diff --git a/libq/Makefile.in b/libq/Makefile.in
index a695113..aa116de 100644
--- a/libq/Makefile.in
+++ b/libq/Makefile.in
@@ -246,21 +246,21 @@ libq_la_LIBADD =
 am__libq_la_SOURCES_DIST = atom.c atom.h basename.c basename.h \
 	busybox.h colors.c colors.h contents.c contents.h copy_file.c \
 	copy_file.h dep.c dep.h eat_file.c eat_file.h hash.c hash.h \
-	human_readable.c human_readable.h i18n.h prelink.c prelink.h \
-	profile.c profile.h rmspace.c rmspace.h safe_io.c safe_io.h \
-	scandirat.c scandirat.h set.c set.h tree.c tree.h xarray.c \
-	xarray.h xasprintf.h xchdir.c xchdir.h xmkdir.c xmkdir.h \
-	xpak.c xpak.h xregex.c xregex.h xsystem.c xsystem.h \
-	hash_md5_sha1.c hash_md5_sha1.h
+	human_readable.c human_readable.h i18n.h move_file.c \
+	move_file.h prelink.c prelink.h profile.c profile.h rmspace.c \
+	rmspace.h safe_io.c safe_io.h scandirat.c scandirat.h set.c \
+	set.h tree.c tree.h xarray.c xarray.h xasprintf.h xchdir.c \
+	xchdir.h xmkdir.c xmkdir.h xpak.c xpak.h xregex.c xregex.h \
+	xsystem.c xsystem.h hash_md5_sha1.c hash_md5_sha1.h
 @QMANIFEST_ENABLED_FALSE@@QTEGRITY_ENABLED_FALSE@am__objects_1 = libq_la-hash_md5_sha1.lo
 am__objects_2 = libq_la-atom.lo libq_la-basename.lo libq_la-colors.lo \
 	libq_la-contents.lo libq_la-copy_file.lo libq_la-dep.lo \
 	libq_la-eat_file.lo libq_la-hash.lo libq_la-human_readable.lo \
-	libq_la-prelink.lo libq_la-profile.lo libq_la-rmspace.lo \
-	libq_la-safe_io.lo libq_la-scandirat.lo libq_la-set.lo \
-	libq_la-tree.lo libq_la-xarray.lo libq_la-xchdir.lo \
-	libq_la-xmkdir.lo libq_la-xpak.lo libq_la-xregex.lo \
-	libq_la-xsystem.lo $(am__objects_1)
+	libq_la-move_file.lo libq_la-prelink.lo libq_la-profile.lo \
+	libq_la-rmspace.lo libq_la-safe_io.lo libq_la-scandirat.lo \
+	libq_la-set.lo libq_la-tree.lo libq_la-xarray.lo \
+	libq_la-xchdir.lo libq_la-xmkdir.lo libq_la-xpak.lo \
+	libq_la-xregex.lo libq_la-xsystem.lo $(am__objects_1)
 am_libq_la_OBJECTS = $(am__objects_2)
 libq_la_OBJECTS = $(am_libq_la_OBJECTS)
 AM_V_lt = $(am__v_lt_@AM_V@)
@@ -290,6 +290,7 @@ am__depfiles_remade = ./$(DEPDIR)/libq_la-atom.Plo \
 	./$(DEPDIR)/libq_la-eat_file.Plo ./$(DEPDIR)/libq_la-hash.Plo \
 	./$(DEPDIR)/libq_la-hash_md5_sha1.Plo \
 	./$(DEPDIR)/libq_la-human_readable.Plo \
+	./$(DEPDIR)/libq_la-move_file.Plo \
 	./$(DEPDIR)/libq_la-prelink.Plo \
 	./$(DEPDIR)/libq_la-profile.Plo \
 	./$(DEPDIR)/libq_la-rmspace.Plo \
@@ -1485,11 +1486,12 @@ top_srcdir = @top_srcdir@
 QFILES = atom.c atom.h basename.c basename.h busybox.h colors.c \
 	colors.h contents.c contents.h copy_file.c copy_file.h dep.c \
 	dep.h eat_file.c eat_file.h hash.c hash.h human_readable.c \
-	human_readable.h i18n.h prelink.c prelink.h profile.c \
-	profile.h rmspace.c rmspace.h safe_io.c safe_io.h scandirat.c \
-	scandirat.h set.c set.h tree.c tree.h xarray.c xarray.h \
-	xasprintf.h xchdir.c xchdir.h xmkdir.c xmkdir.h xpak.c xpak.h \
-	xregex.c xregex.h xsystem.c xsystem.h $(NULL) $(am__append_1)
+	human_readable.h i18n.h move_file.c move_file.h prelink.c \
+	prelink.h profile.c profile.h rmspace.c rmspace.h safe_io.c \
+	safe_io.h scandirat.c scandirat.h set.c set.h tree.c tree.h \
+	xarray.c xarray.h xasprintf.h xchdir.c xchdir.h xmkdir.c \
+	xmkdir.h xpak.c xpak.h xregex.c xregex.h xsystem.c xsystem.h \
+	$(NULL) $(am__append_1)
 noinst_LTLIBRARIES = libq.la
 libq_la_SOURCES = $(QFILES)
 libq_la_CPPFLAGS = \
@@ -1561,6 +1563,7 @@ distclean-compile:
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-hash.Plo@am__quote@ # am--include-marker
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-hash_md5_sha1.Plo@am__quote@ # am--include-marker
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-human_readable.Plo@am__quote@ # am--include-marker
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-move_file.Plo@am__quote@ # am--include-marker
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-prelink.Plo@am__quote@ # am--include-marker
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-profile.Plo@am__quote@ # am--include-marker
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-rmspace.Plo@am__quote@ # am--include-marker
@@ -1665,6 +1668,13 @@ libq_la-human_readable.lo: human_readable.c
 @AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
 @am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libq_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libq_la-human_readable.lo `test -f 'human_readable.c' || echo '$(srcdir)/'`human_readable.c
 
+libq_la-move_file.lo: move_file.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libq_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libq_la-move_file.lo -MD -MP -MF $(DEPDIR)/libq_la-move_file.Tpo -c -o libq_la-move_file.lo `test -f 'move_file.c' || echo '$(srcdir)/'`move_file.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libq_la-move_file.Tpo $(DEPDIR)/libq_la-move_file.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='move_file.c' object='libq_la-move_file.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libq_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libq_la-move_file.lo `test -f 'move_file.c' || echo '$(srcdir)/'`move_file.c
+
 libq_la-prelink.lo: prelink.c
 @am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libq_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libq_la-prelink.lo -MD -MP -MF $(DEPDIR)/libq_la-prelink.Tpo -c -o libq_la-prelink.lo `test -f 'prelink.c' || echo '$(srcdir)/'`prelink.c
 @am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libq_la-prelink.Tpo $(DEPDIR)/libq_la-prelink.Plo
@@ -1903,6 +1913,7 @@ distclean: distclean-am
 	-rm -f ./$(DEPDIR)/libq_la-hash.Plo
 	-rm -f ./$(DEPDIR)/libq_la-hash_md5_sha1.Plo
 	-rm -f ./$(DEPDIR)/libq_la-human_readable.Plo
+	-rm -f ./$(DEPDIR)/libq_la-move_file.Plo
 	-rm -f ./$(DEPDIR)/libq_la-prelink.Plo
 	-rm -f ./$(DEPDIR)/libq_la-profile.Plo
 	-rm -f ./$(DEPDIR)/libq_la-rmspace.Plo
@@ -1971,6 +1982,7 @@ maintainer-clean: maintainer-clean-am
 	-rm -f ./$(DEPDIR)/libq_la-hash.Plo
 	-rm -f ./$(DEPDIR)/libq_la-hash_md5_sha1.Plo
 	-rm -f ./$(DEPDIR)/libq_la-human_readable.Plo
+	-rm -f ./$(DEPDIR)/libq_la-move_file.Plo
 	-rm -f ./$(DEPDIR)/libq_la-prelink.Plo
 	-rm -f ./$(DEPDIR)/libq_la-profile.Plo
 	-rm -f ./$(DEPDIR)/libq_la-rmspace.Plo

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     117869a1b0d19c5e3a6b59e9e43e6849ed92efd4
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Dec 26 13:48:49 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Dec 26 13:48:49 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=117869a1

libq/move_file: code to move a file or copy+remove it

This code originated from qmerge.c, it was adapted to be more generic in
a function.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/Makefile.am |   1 +
 libq/move_file.c | 105 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 libq/move_file.h |  21 +++++++++++
 3 files changed, 127 insertions(+)

diff --git a/libq/Makefile.am b/libq/Makefile.am
index da100b6..c0402a4 100644
--- a/libq/Makefile.am
+++ b/libq/Makefile.am
@@ -11,6 +11,7 @@ QFILES = \
 	hash.c hash.h \
 	human_readable.c human_readable.h \
 	i18n.h \
+	move_file.c move_file.h \
 	prelink.c prelink.h \
 	profile.c profile.h \
 	rmspace.c rmspace.h \

diff --git a/libq/move_file.c b/libq/move_file.c
new file mode 100644
index 0000000..07cf69f
--- /dev/null
+++ b/libq/move_file.c
@@ -0,0 +1,105 @@
+/*
+ * Copyright 2005-2021 Gentoo Authors
+ * Distributed under the terms of the GNU General Public License v2
+ *
+ * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
+ * Copyright 2005-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2018-     Fabian Groffen  - <grobian@gentoo.org>
+ */
+
+#include "main.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include "stat-time.h"
+
+#include "copy_file.h"
+#include "move_file.h"
+
+int
+move_file(int rootfd_src, const char *name_src,
+		  int rootfd_dst, const char *name_dst,
+		  struct stat *stat_src)
+{
+	/* first try fast path -- src/dst are same device, else
+	 * fall back to slow path -- manual read/write */
+	if (renameat(rootfd_src, name_src, rootfd_dst, name_dst) != 0) {
+		int             fd_src;
+		int             fd_dst;
+		char            tmpname_dst[_Q_PATH_MAX];
+		struct stat     st;
+		struct timespec times[2];
+
+		fd_src = openat(rootfd_src, name_src, O_RDONLY|O_CLOEXEC);
+		if (fd_src < 0) {
+			warnp("could not read source file %s", name_src);
+			return fd_src;
+		}
+
+		if (stat_src == NULL) {
+			if (fstat(fd_src, &st) != 0) {
+				warnp("could not stat source file %s", name_src);
+				return -1;
+			}
+
+			stat_src = &st;
+		}
+
+		/* do not write the file in place ...
+	 	 * will fail with files that are in use
+	 	 * plus it isn't atomic, so we could leave a mess */
+	 	snprintf(tmpname_dst, sizeof(tmpname_dst), ".%u.%s",
+	 			 getpid(), name_dst);
+		fd_dst = openat(rootfd_dst, tmpname_dst,
+					 	O_WRONLY|O_CLOEXEC|O_CREAT|O_TRUNC,
+					 	stat_src->st_mode);
+		if (fd_dst < 0) {
+			warnp("could not open destination file %s (for %s)",
+				  tmpname_dst, name_dst);
+			close(fd_src);
+			return fd_dst;
+		}
+
+		/* make sure owner/mode is sane before we write out data */
+		if (fchown(fd_dst, stat_src->st_uid, stat_src->st_gid) != 0) {
+			warnp("could not set ownership (%zu/%zu) for %s",
+			  	  (size_t)stat_src->st_uid, (size_t)stat_src->st_gid, name_dst);
+			return -1;
+		}
+		if (fchmod(fd_dst, stat_src->st_mode) != 0) {
+			warnp("could not set permission (%u) for %s",
+			  	  (int)stat_src->st_mode, name_dst);
+			return -1;
+		}
+
+		/* do the actual data copy */
+		if (copy_file_fd(fd_src, fd_dst)) {
+			warnp("could not write to file %s", name_dst);
+			if (unlinkat(rootfd_dst, tmpname_dst, 0) != 0) {
+				/* don't care */;
+			}
+			close(fd_src);
+			close(fd_dst);
+			return -1;
+		}
+
+		/* Preserve the file times */
+		times[0] = get_stat_atime(&st);
+		times[1] = get_stat_mtime(&st);
+		futimens(fd_dst, times);
+
+		close(fd_src);
+		close(fd_dst);
+
+		/* finally move the new tmp dst file to the right place, which
+		 * should be on the same FS/device now */
+		if (renameat(rootfd_dst, tmpname_dst, rootfd_dst, name_dst)) {
+			warnp("could not rename %s to %s", tmpname_dst, name_dst);
+			return -1;
+		}
+	}
+
+	return 0;
+}

diff --git a/libq/move_file.h b/libq/move_file.h
new file mode 100644
index 0000000..a454165
--- /dev/null
+++ b/libq/move_file.h
@@ -0,0 +1,21 @@
+/*
+ * Copyright 2005-2021 Gentoo Authors
+ * Distributed under the terms of the GNU General Public License v2
+ *
+ * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
+ * Copyright 2005-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2018-     Fabian Groffen  - <grobian@gentoo.org>
+ */
+
+#ifndef _MOVE_FILE_H
+
+#define _MOVE_FILE_H 1
+
+#include <sys/types.h>
+#include <sys/stat.h>
+
+int move_file(int rootfd_src, const char *name_src,
+			  int rootfd_dst, const char *name_dst,
+		  	  struct stat *stat_src);
+
+#endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     8165933c5b9986b536cf91b8ba82f7e76c9cc758
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Dec 26 13:54:21 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Dec 26 13:54:21 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=8165933c

libq/move_file: remove source file when done

move suggests the source is gone, so make sure it is :)

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/move_file.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/libq/move_file.c b/libq/move_file.c
index 07cf69f..b98c8e2 100644
--- a/libq/move_file.c
+++ b/libq/move_file.c
@@ -85,7 +85,7 @@ move_file(int rootfd_src, const char *name_src,
 			return -1;
 		}
 
-		/* Preserve the file times */
+		/* preserve the file times */
 		times[0] = get_stat_atime(&st);
 		times[1] = get_stat_mtime(&st);
 		futimens(fd_dst, times);
@@ -93,12 +93,15 @@ move_file(int rootfd_src, const char *name_src,
 		close(fd_src);
 		close(fd_dst);
 
-		/* finally move the new tmp dst file to the right place, which
+		/* move the new tmp dst file to the right place, which
 		 * should be on the same FS/device now */
 		if (renameat(rootfd_dst, tmpname_dst, rootfd_dst, name_dst)) {
 			warnp("could not rename %s to %s", tmpname_dst, name_dst);
 			return -1;
 		}
+
+		/* finally remove the source file */
+		return unlinkat(rootfd_src, name_src, 0);
 	}
 
 	return 0;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     53d7d24872527a69501f4c74a44e16e29aa3bb4a
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Dec 13 07:17:04 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Dec 13 07:17:04 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=53d7d248

libq/copy_file: employ sendfile() if possible

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/copy_file.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 67 insertions(+), 11 deletions(-)

diff --git a/libq/copy_file.c b/libq/copy_file.c
index e4619ce..955fd78 100644
--- a/libq/copy_file.c
+++ b/libq/copy_file.c
@@ -1,29 +1,85 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2021 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2011-2016 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2021-     Fabian Groffen  - <grobian@gentoo.org>
  */
 
 #include "main.h"
 #include "safe_io.h"
 #include "copy_file.h"
 
+/* includes for when sendfile is available */
+#include <fcntl.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#if defined(HAVE_SENDFILE4_SUPPORT)
+/* Linux/Solaris */
+# include <sys/sendfile.h>
+#elif defined(HAVE_SENDFILE6_SUPPORT) || defined(HAVE_SENDFILE7_SUPPORT)
+/* macOS (since Darwin 9) + FreeBSD */
+# include <sys/socket.h>
+# include <sys/uio.h>
+#endif
+
 int copy_file_fd(int fd_src, int fd_dst)
 {
-	ssize_t rcnt, wcnt;
-	char buf[64 * 1024];
+#if defined(HAVE_SENDFILE4_SUPPORT) || \
+	defined(HAVE_SENDFILE6_SUPPORT) || \
+	defined(HAVE_SENDFILE7_SUPPORT)
+	struct stat stat_buf;
+	ssize_t     ret;
+	size_t      len;
+	off_t       offset = 0;
 
-	while (1) {
-		rcnt = safe_read(fd_src, buf, sizeof(buf));
-		if (rcnt < 0)
-			return -1;
-		else if (rcnt == 0)
+	if (fstat(fd_src, &stat_buf) != -1) {
+		len = (size_t)stat_buf.st_size;
+
+#if defined(HAVE_SENDFILE4_SUPPORT)
+		/* Linux/Solaris */
+		ret = sendfile(fd_dst, fd_src, &offset, len);
+		/* everything looks fine, return success */
+		if (ret == (ssize_t)len)
 			return 0;
+#elif defined(HAVE_SENDFILE6_SUPPORT)
+		/* macOS (since Darwin 9) */
+		offset = len;
+		ret = (ssize_t)sendfile(fd_src, fd_dst, 0, &offset, NULL, 0);
+		/* everything looks fine, return success */
+		if (offset == (off_t)len)
+			return 0;
+#elif defined(HAVE_SENDFILE7_SUPPORT)
+		/* FreeBSD */
+		ret = (ssize_t)sendfile(fd_src, fd_dst, offset, len, NULL, &offset, 0);
+		/* everything looks fine, return success */
+		if (offset == (off_t)len)
+			return 0;
+#endif
 
-		wcnt = safe_write(fd_dst, buf, rcnt);
-		if (wcnt == -1)
-			return -1;
+		/* fall back to read/write, rewind the fd */
+		lseek(fd_src, 0, SEEK_SET);
+	}
+#endif /* HAVE_SENDFILE */
+
+	/* fallback, keep in its own scope, so we avoid 64K stack alloc if
+	 * sendfile works properly */
+	{
+		ssize_t rcnt, wcnt;
+		char buf[64 * 1024];
+
+		while (1) {
+			rcnt = safe_read(fd_src, buf, sizeof(buf));
+			if (rcnt < 0)
+				return -1;
+			else if (rcnt == 0)
+				return 0;
+
+			wcnt = safe_write(fd_dst, buf, rcnt);
+			if (wcnt == -1)
+				return -1;
+		}
 	}
 }
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     83eaa405a97e45e8cdbf20ad1bb30e82c902d7e2
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Dec 13 07:22:30 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Dec 13 07:22:30 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=83eaa405

libq/set: allow array_set to be used with an empty set

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/set.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libq/set.c b/libq/set.c
index 88b5876..6c9fae0 100644
--- a/libq/set.c
+++ b/libq/set.c
@@ -271,6 +271,11 @@ array_set(set *q, array_t *ret)
 	array_t blank = array_init_decl;
 
 	*ret = blank;
+
+	/* allow using empty set */
+	if (q == NULL)
+		return 0;
+
 	for (i = 0; i < _SET_HASH_SIZE; i++) {
 		for (w = q->buckets[i]; w != NULL; w = w->next)
 			xarraypush_ptr(ret, w->name);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     a9872a3fc8b489e5dd96bcaaa57d02738bf6e077
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Nov 13 14:26:00 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Nov 13 14:26:00 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=a9872a3f

libq/dep: fix inversed logic in dep_prune_use

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/dep.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libq/dep.c b/libq/dep.c
index 3667ae5..d431367 100644
--- a/libq/dep.c
+++ b/libq/dep.c
@@ -319,7 +319,8 @@ dep_prune_use(dep_node *root, set *use)
 		dep_prune_use(root->neighbor, use);
 	if (root->type == DEP_USE) {
 		bool invert = (root->info[0] == '!' ? 1 : 0);
-		bool notfound = contains_set(root->info + (invert ? 1 : 0), use);
+		bool notfound =
+			contains_set(root->info + (invert ? 1 : 0), use) == NULL;
 
 		if (notfound ^ invert) {
 			root->type = DEP_NULL;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     023d4496ef445a2f6f05b9c288e9816695d6daf4
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Oct  9 12:12:11 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Oct  9 12:12:11 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=023d4496

libq/atom: perform correct SUBSLOT matching in compare

fixup after ef14d5f7bb09b8a90e827262798ebd1fde58913a
now SUBSLOT is never NULL, ensure we check it was explicitly set or not,
which is what used to be NULL.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 9a51e22..0b5fcdd 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -572,7 +572,9 @@ atom_compare_flg(const depend_atom *data, const depend_atom *query, int flags)
 			if (query->SLOT == NULL && data->SLOT == NULL)
 				return NOT_EQUAL;
 			if (query->SLOT != NULL) {
-				if (query->SUBSLOT == NULL || flags & ATOM_COMP_NOSUBSLOT) {
+				if (query->SUBSLOT == query->SLOT ||
+						flags & ATOM_COMP_NOSUBSLOT)
+				{
 					/* ^perl:0 -> match different SLOT */
 					if (data->SLOT == NULL ||
 							strcmp(query->SLOT, data->SLOT) == 0)
@@ -583,7 +585,7 @@ atom_compare_flg(const depend_atom *data, const depend_atom *query, int flags)
 							strcmp(query->SLOT, data->SLOT) != 0)
 						return NOT_EQUAL;
 					if (!(flags & ATOM_COMP_NOSUBSLOT))
-						if (data->SUBSLOT == NULL ||
+						if (data->SUBSLOT == query->SLOT ||
 								strcmp(query->SUBSLOT, data->SUBSLOT) == 0)
 							return NOT_EQUAL;
 				}
@@ -601,8 +603,8 @@ atom_compare_flg(const depend_atom *data, const depend_atom *query, int flags)
 				if (bl_op == ATOM_BL_NONE)
 					return NOT_EQUAL;
 			} else if (!(flags & ATOM_COMP_NOSUBSLOT)) {
-				if (query->SUBSLOT != NULL) {
-					if (data->SUBSLOT == NULL) {
+				if (query->SUBSLOT != query->SLOT) {
+					if (data->SUBSLOT == data->SLOT) {
 						if (bl_op == ATOM_BL_NONE)
 							return NOT_EQUAL;
 					} else {

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     ef14d5f7bb09b8a90e827262798ebd1fde58913a
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Oct  4 06:28:02 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Oct  4 06:28:02 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ef14d5f7

libq/atom: parse/set SUBSLOT when absent to SLOT (PMS 7.2)

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index d210ed6..9a51e22 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -218,6 +218,10 @@ atom_explode_cat(const char *atom, const char *cat)
 		/* set to NULL if there's nothing */
 		if (ret->SLOT[0] == '\0')
 			ret->SLOT = NULL;
+
+		/* PMS 7.2: SUBSLOT defaults to SLOT when unset */
+		if (ret->SUBSLOT == NULL)
+			ret->SUBSLOT = ret->SLOT;
 	}
 
 	/* see if we have any suffix operators */
@@ -380,7 +384,7 @@ atom_clone(depend_atom *atom)
 		rlen = strlen(atom->REPO) + 1;
 	if (atom->SLOT != NULL)
 		slen = strlen(atom->SLOT) + 1;
-	if (atom->SUBSLOT != NULL)
+	if (atom->SUBSLOT != NULL && atom->SUBSLOT != atom->SLOT)
 		sslen = strlen(atom->SUBSLOT) + 1;
 	if (atom->CATEGORY != NULL)
 		clen = strlen(atom->CATEGORY) + 1;
@@ -427,9 +431,13 @@ atom_clone(depend_atom *atom)
 		p += slen;
 	}
 	if (atom->SUBSLOT != NULL) {
-		ret->SUBSLOT = p;
-		memcpy(ret->SUBSLOT, atom->SUBSLOT, sslen);
-		p += sslen;
+		if (atom->SUBSLOT == atom->SLOT) {  /* PMS 7.2 */
+			ret->SUBSLOT = ret->SLOT;
+		} else {
+			ret->SUBSLOT = p;
+			memcpy(ret->SUBSLOT, atom->SUBSLOT, sslen);
+			p += sslen;
+		}
 	}
 	if (atom->REPO != NULL) {
 		ret->REPO = p;
@@ -820,7 +828,8 @@ atom_to_string_r(char *buf, size_t buflen, depend_atom *a)
 	if (a->SLOT != NULL || a->slotdep != ATOM_SD_NONE)
 		off += snprintf(buf + off, buflen - off, ":%s%s%s%s",
 				a->SLOT ? a->SLOT : "",
-				a->SUBSLOT ? "/" : "", a->SUBSLOT ? a->SUBSLOT : "",
+				a->SUBSLOT && a->SUBSLOT != a->SLOT ?  "/" : "",
+				a->SUBSLOT && a->SUBSLOT != a->SLOT ? a->SUBSLOT : "",
 				atom_slotdep_str[a->slotdep]);
 	for (ud = a->usedeps; ud != NULL; ud = ud->next)
 		off += snprintf(buf + off, buflen - off, "%s%s%s%s%s",
@@ -933,7 +942,8 @@ atom_format_r(
 								HN(atom->SLOT),
 								NORM);
 				} else if (!strncmp("SUBSLOT", fmt, len)) {
-					if (showit || atom->SUBSLOT)
+					if (showit ||
+							(atom->SUBSLOT && atom->SUBSLOT != atom->SLOT))
 						append_buf(buf, buflen, "%s%s%s%s%s",
 								YELLOW,
 								connected ? "/" : "",

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     48ca7946e74ed6c1c065589c50327948a41c25b7
Author:     Barnabás Virágh <cyborgyn <AT> gmail <DOT> com>
AuthorDate: Mon Oct  4 06:09:12 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Oct  4 06:09:12 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=48ca7946

libq/tree: properly set SUBSLOT value when absent (PMS 7.2)

Bug: https://bugs.gentoo.org/816060
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index 847a343..87df175 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1613,8 +1613,13 @@ tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete)
 			char *ptr;
 			if ((ptr = strchr(pkg_ctx->atom->SLOT, '/')) != NULL) {
 				*ptr++ = '\0';
-				pkg_ctx->atom->SUBSLOT = ptr;
+			} else {
+				/* PMS 7.2: When the sub-slot part is omitted from the
+				 * SLOT definition, the package is considered to have an
+				 * implicit sub-slot which is equal to the regular slot. */
+				ptr = pkg_ctx->atom->SLOT;
 			}
+			pkg_ctx->atom->SUBSLOT = ptr;
 		}
 	}
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     c7e750202c5e17bc8089a8777c4d433b2f63e531
Author:     Barnabás Virágh <cyborgyn <AT> gmail <DOT> com>
AuthorDate: Sun Oct  3 10:47:50 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Oct  3 10:47:50 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=c7e75020

libq/atom: fix atom_clone wrt SUBSLOT

A thinko that assumed SUBSLOT was stored in SLOT (which it usually is)
caused it to never be copied, since obviously SLOT is terminated, so
SUBSLOT, even when originally part of SLOT, would never be contained
within SLOT.

Just copy it separately, which has the additional bonus of being less
obscure.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 0959be5..d210ed6 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -373,12 +373,15 @@ atom_clone(depend_atom *atom)
 	size_t plen = 0;
 	size_t nlen = 0;
 	size_t slen = 0;
+	size_t sslen = 0;
 	size_t rlen = 0;
 
 	if (atom->REPO != NULL)
 		rlen = strlen(atom->REPO) + 1;
 	if (atom->SLOT != NULL)
 		slen = strlen(atom->SLOT) + 1;
+	if (atom->SUBSLOT != NULL)
+		sslen = strlen(atom->SUBSLOT) + 1;
 	if (atom->CATEGORY != NULL)
 		clen = strlen(atom->CATEGORY) + 1;
 	if (atom->PF != NULL)
@@ -388,7 +391,7 @@ atom_clone(depend_atom *atom)
 	if (atom->PN != NULL)
 		nlen = strlen(atom->PN) + 1;
 
-	alen = sizeof(*ret) + clen + flen + plen + nlen + rlen + slen;
+	alen = sizeof(*ret) + clen + flen + plen + nlen + rlen + slen + sslen;
 	ret = xmalloc(alen);
 	memset(ret, '\0', sizeof(*ret));
 
@@ -423,8 +426,11 @@ atom_clone(depend_atom *atom)
 		memcpy(ret->SLOT, atom->SLOT, slen);
 		p += slen;
 	}
-	if (atom->SUBSLOT > atom->SLOT && atom->SUBSLOT < (atom->SLOT + slen))
-		ret->SUBSLOT = ret->SLOT + (atom->SUBSLOT - atom->SLOT);
+	if (atom->SUBSLOT != NULL) {
+		ret->SUBSLOT = p;
+		memcpy(ret->SUBSLOT, atom->SUBSLOT, sslen);
+		p += sslen;
+	}
 	if (atom->REPO != NULL) {
 		ret->REPO = p;
 		memcpy(ret->REPO, atom->REPO, rlen);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e0639a1f8654092654520762b58a122506a1fa72
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Jun 23 07:12:20 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Jun 23 07:12:20 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e0639a1f

libq/tree: correct macro for tree_foreach_pkg

drop the trailing ; to allow the call to be embedded

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libq/tree.h b/libq/tree.h
index 0aa534d..052715b 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -154,9 +154,9 @@ void tree_close_pkg(tree_pkg_ctx *pkg_ctx);
 int tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 		bool sort, const depend_atom *query);
 #define tree_foreach_pkg_fast(ctx, cb, priv, query) \
-	tree_foreach_pkg(ctx, cb, priv, false, query);
+	tree_foreach_pkg(ctx, cb, priv, false, query)
 #define tree_foreach_pkg_sorted(ctx, cb, priv, query) \
-	tree_foreach_pkg(ctx, cb, priv, true, query);
+	tree_foreach_pkg(ctx, cb, priv, true, query)
 set *tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms);
 depend_atom *tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete);
 tree_match_ctx *tree_match_atom(tree_ctx *t, const depend_atom *q, int flags);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     9a4c654be5e759b4bb0fed1ac802a57fb1cb5c30
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Jun  8 15:28:47 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue Jun  8 15:28:47 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=9a4c654b

libq/tree: build cache for Packages/binpkgs

Because we cannot trust Packages file to be in sorted order, and the
order isn't what we want (most recent match on top), we have to sort the
pkgs found in Packages, so make that exploit the cache, which will speed
up any subsequent tree_match_atom calls, which is good.

Do the same for binpkg trees, since they also cannot be cached
otherwise.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 180 ++++++++++++++++++++++++++++++------------------------------
 libq/tree.h |   1 +
 2 files changed, 90 insertions(+), 91 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 358b1f2..d7ec882 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -312,8 +312,29 @@ tree_next_cat(tree_ctx *ctx)
 
 	if (ctx->do_sort) {
 		if (ctx->cat_de == NULL) {
-			ctx->cat_cnt = scandirat(ctx->tree_fd,
-					".", &ctx->cat_de, tree_filter_cat, alphasort);
+			if (ctx->cache.all_categories) {
+				char **cats;
+				size_t i;
+				size_t len;
+				struct dirent **ret;
+
+				/* exploit the cache instead of reading from directory */
+				ctx->cat_cnt = cnt_set(ctx->cache.categories);
+				ctx->cat_de = ret = xmalloc(sizeof(*ret) * ctx->cat_cnt);
+				list_set(ctx->cache.categories, &cats);
+				for (i = 0; i < ctx->cat_cnt; i++) {
+					len = strlen(cats[i]) + 1;
+					ret[i] = xzalloc(sizeof(*de) + len);
+					snprintf(ret[i]->d_name, len, "%s", cats[i]);
+				}
+				if (i > 1)
+					qsort(ret, ctx->cat_cnt, sizeof(*ret),
+							(int (*)(const void *, const void *))alphasort);
+				free(cats);
+			} else {
+				ctx->cat_cnt = scandirat(ctx->tree_fd,
+						".", &ctx->cat_de, tree_filter_cat, alphasort);
+			}
 			ctx->cat_cur = 0;
 		}
 
@@ -1511,7 +1532,7 @@ tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 	ctx->cat_de = NULL;
 	ctx->cat_cur = 0;
 	ctx->cat_cnt = 0;
-	ctx->do_sort = 0;
+	ctx->do_sort = false;
 
 	return ret;
 }
@@ -1624,77 +1645,33 @@ tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms)
 	return state.cpf;
 }
 
-struct tree_match_pkgs_cb_ctx {
-	int flags;
-	tree_match_ctx *ret;
-};
-
 static int
-tree_match_atom_packages_cb(tree_pkg_ctx *ctx, void *priv)
+tree_match_atom_cache_populate_cb(tree_pkg_ctx *ctx, void *priv)
 {
-	struct tree_match_pkgs_cb_ctx *rctx = priv;
-	depend_atom *a;
-	tree_match_ctx *n;
-
-	/* skip anything after finding first match */
-	if (rctx->flags & TREE_MATCH_FIRST && rctx->ret != NULL)
-		return 1;
-
-	a = tree_get_atom(ctx, rctx->flags & TREE_MATCH_FULL_ATOM);
-
-	/* skip virtual category if not requested */
-	if (!(rctx->flags & TREE_MATCH_VIRTUAL) &&
-			strcmp(a->CATEGORY, "virtual") == 0)
-		return 1;
-
-	n = xzalloc(sizeof(tree_match_ctx));
-	n->free_atom = true;
-	n->atom = atom_clone(a);
-	snprintf(n->path, sizeof(n->path), "%s/%s/%s.tbz2",
-			(char *)ctx->cat_ctx->ctx->path, ctx->cat_ctx->name, ctx->name);
-	if (rctx->flags & TREE_MATCH_METADATA) {
-		n->meta = xmalloc(sizeof(*n->meta));
-		/* for Packages, all pointers to meta here are to the in memory
-		 * copy of the Packages file, so these pointers can just be
-		 * copied since the tree has to remain open, thus the pointers
-		 * will stay valid */
-		memcpy(n->meta, ctx->cat_ctx->ctx->pkgs, sizeof(*n->meta));
+	set *cache = priv;
+	tree_cat_ctx *cat_ctx;
+	tree_pkg_ctx *pkg;
+	tree_ctx *tctx = ctx->cat_ctx->ctx;
+	depend_atom *atom = tree_get_atom(ctx, true);
+
+	(void)priv;
+
+	cat_ctx = get_set(atom->CATEGORY, cache);
+	if (cat_ctx == NULL) {
+		cat_ctx = tree_open_cat(tctx, ".");
+		add_set_value(atom->CATEGORY, cat_ctx, cache);
+		/* get a pointer from the set */
+		cat_ctx->name = contains_set(atom->CATEGORY, cache);
 	}
 
-	n->next = rctx->ret;
-	rctx->ret = n;
-
-	return 0;
-}
-
-static int
-tree_match_atom_binpkg_cb(tree_pkg_ctx *ctx, void *priv)
-{
-	struct tree_match_pkgs_cb_ctx *rctx = priv;
-	depend_atom *a;
-	tree_match_ctx *n;
-
-	/* skip anything after finding first match */
-	if (rctx->flags & TREE_MATCH_FIRST && rctx->ret != NULL)
-		return 1;
-
-	a = tree_get_atom(ctx, rctx->flags & TREE_MATCH_FULL_ATOM);
-
-	/* skip virtual category if not requested */
-	if (!(rctx->flags & TREE_MATCH_VIRTUAL) &&
-			strcmp(a->CATEGORY, "virtual") == 0)
-		return 1;
-
-	n = xzalloc(sizeof(tree_match_ctx));
-	n->free_atom = true;
-	n->atom = atom_clone(a);
-	snprintf(n->path, sizeof(n->path), "%s/%s/%s.tbz2",
-			(char *)ctx->cat_ctx->ctx->path, ctx->cat_ctx->name, ctx->name);
-	if (rctx->flags & TREE_MATCH_METADATA)
-		n->meta = tree_pkg_read(ctx);
-
-	n->next = rctx->ret;
-	rctx->ret = n;
+	/* FIXME: this really could use a set */
+	cat_ctx->pkg_cnt++;
+	cat_ctx->pkg_ctxs = xrealloc(cat_ctx->pkg_ctxs,
+			sizeof(*cat_ctx->pkg_ctxs) * cat_ctx->pkg_cnt);
+	cat_ctx->pkg_ctxs[cat_ctx->pkg_cnt - 1] =
+		pkg = tree_open_pkg(cat_ctx, atom->PF);
+	pkg->atom = atom_clone(atom);
+	pkg->name = pkg->atom->PF;
 
 	return 0;
 }
@@ -1707,24 +1684,48 @@ tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 	tree_match_ctx *ret = NULL;
 	depend_atom *atom;
 
-	if (ctx->cachetype == CACHE_PACKAGES) {
-		struct tree_match_pkgs_cb_ctx rctx;
-		/* Packages needs to be serviced separately because it doesn't
-		 * use a tree internally, but reads off of the Packages file */
-		rctx.flags = flags;
-		rctx.ret = NULL;
-		ctx->query_atom = query;
-		tree_foreach_packages(ctx, tree_match_atom_packages_cb, &rctx);
-		ctx->query_atom = NULL;
-		return rctx.ret;
-	} else if (ctx->cachetype == CACHE_BINPKGS) {
-		struct tree_match_pkgs_cb_ctx rctx;
-		/* this sulks, but binpkgs modify the pkg_ctx->name to strip off
-		 * .tbz2, and that makes it non-reusable */
-		rctx.flags = flags;
-		rctx.ret = NULL;
-		tree_foreach_pkg(ctx, tree_match_atom_binpkg_cb, &rctx, true, query);
-		return rctx.ret;
+	ctx->do_sort = true;     /* sort uses buffer, which cache relies on */
+	ctx->query_atom = NULL;  /* ensure the cache contains ALL pkgs */
+
+	if ((ctx->cachetype == CACHE_PACKAGES || ctx->cachetype == CACHE_BINPKGS)
+			&& ctx->cache.categories == NULL)
+	{
+		set *cache;
+		DECLARE_ARRAY(cats);
+		size_t n;
+
+		/* Reading these formats requires us to traverse the tree for
+		 * each operation, so we better cache the entire thing in one
+		 * go.  Packages in addition may not store the atoms sorted
+		 * (it's a file with content after all) and since we rely on
+		 * this for latest and first match, it is important that we sort
+		 * the contents so we're sure */
+
+		cache = ctx->cache.categories;
+		if (ctx->cache.categories != NULL)
+			ctx->cache.categories = NULL;
+		else
+			cache = create_set();
+
+		if (ctx->cachetype == CACHE_PACKAGES)
+			tree_foreach_packages(ctx,
+					tree_match_atom_cache_populate_cb, cache);
+		else  /* BINPKG */
+			tree_foreach_pkg(ctx,
+					tree_match_atom_cache_populate_cb, cache, true, NULL);
+		ctx->do_sort = true;  /* turn it back on */
+		ctx->cache.all_categories = true;
+
+		ctx->cache.categories = cache;
+
+		/* loop through all categories, and sort the pkgs */
+		values_set(cache, cats);
+		array_for_each(cats, n, cat_ctx) {
+			if (cat_ctx->pkg_cnt > 1) {
+				qsort(cat_ctx->pkg_ctxs, cat_ctx->pkg_cnt,
+						sizeof(*cat_ctx->pkg_ctxs), tree_pkg_compar);
+			}
+		}
 	}
 
 	/* activate cache for future lookups, tree_match_atom relies on
@@ -1733,9 +1734,6 @@ tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 	if (ctx->cache.categories == NULL)
 		ctx->cache.categories = create_set();
 
-	ctx->do_sort = true;     /* sort uses buffer, which cache relies on */
-	ctx->query_atom = NULL;  /* ensure the cache contains ALL pkgs */
-
 #define search_cat(C) \
 { \
 	char *lastpn = NULL; \

diff --git a/libq/tree.h b/libq/tree.h
index 5a12c92..8741bad 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -48,6 +48,7 @@ struct tree_ctx {
 	depend_atom *query_atom;
 	struct tree_cache {
 		set *categories;
+		bool all_categories:1;
 	} cache;
 };
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     493ec668848a3f7dc74001454957205ff9017879
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Jun 12 13:56:20 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Jun 12 13:56:20 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=493ec668

libq/tree: better tree_match interaction with binpkgs

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 41 ++++++++++++++++++++++++++++++++---------
 1 file changed, 32 insertions(+), 9 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index d7ec882..2180867 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1080,6 +1080,10 @@ static tree_pkg_meta *
 tree_pkg_read(tree_pkg_ctx *pkg_ctx)
 {
 	tree_ctx *ctx = pkg_ctx->cat_ctx->ctx;
+	tree_pkg_meta *ret = NULL;
+
+	if (pkg_ctx->meta != NULL)
+		return pkg_ctx->meta;
 
 	if (pkg_ctx->fd == -1) {
 		if (ctx->cachetype == CACHE_EBUILD || ctx->cachetype == CACHE_BINPKGS) {
@@ -1098,19 +1102,22 @@ tree_pkg_read(tree_pkg_ctx *pkg_ctx)
 	}
 
 	if (ctx->cachetype == CACHE_METADATA_MD5) {
-		return tree_read_file_md5(pkg_ctx);
+		ret = tree_read_file_md5(pkg_ctx);
 	} else if (ctx->cachetype == CACHE_METADATA_PMS) {
-		return tree_read_file_pms(pkg_ctx);
+		ret = tree_read_file_pms(pkg_ctx);
 	} else if (ctx->cachetype == CACHE_EBUILD) {
-		return tree_read_file_ebuild(pkg_ctx);
+		ret = tree_read_file_ebuild(pkg_ctx);
 	} else if (ctx->cachetype == CACHE_BINPKGS) {
-		return tree_read_file_binpkg(pkg_ctx);
+		ret = tree_read_file_binpkg(pkg_ctx);
 	} else if (ctx->cachetype == CACHE_PACKAGES) {
-		return (tree_pkg_meta *)pkg_ctx->cat_ctx->ctx->pkgs;
+		ret = (tree_pkg_meta *)pkg_ctx->cat_ctx->ctx->pkgs;
 	}
 
-	warn("Unknown/unsupported metadata cache type!");
-	return NULL;
+	pkg_ctx->meta = ret;
+
+	if (ret == NULL)
+		warn("Unknown/unsupported metadata cache type!");
+	return ret;
 }
 
 static void
@@ -1204,6 +1211,12 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 		if (*key == NULL && ctx->cachetype == CACHE_PACKAGES) {
 			ctx->cachetype = CACHE_BINPKGS;
 			pkg_ctx->fd = -1;
+
+			/* trigger tree_pkg_read to do something */
+			if ((void *)pkg_ctx->meta != (void *)pkg_ctx->cat_ctx->ctx->pkgs)
+				free(pkg_ctx->meta);
+			pkg_ctx->meta = NULL;
+
 			pkg_ctx->meta = tree_pkg_read(pkg_ctx);
 			ctx->cachetype = CACHE_PACKAGES;
 			if (pkg_ctx->meta == NULL) {
@@ -1653,6 +1666,7 @@ tree_match_atom_cache_populate_cb(tree_pkg_ctx *ctx, void *priv)
 	tree_pkg_ctx *pkg;
 	tree_ctx *tctx = ctx->cat_ctx->ctx;
 	depend_atom *atom = tree_get_atom(ctx, true);
+	tree_pkg_meta *meta = tree_pkg_read(ctx);
 
 	(void)priv;
 
@@ -1671,7 +1685,15 @@ tree_match_atom_cache_populate_cb(tree_pkg_ctx *ctx, void *priv)
 	cat_ctx->pkg_ctxs[cat_ctx->pkg_cnt - 1] =
 		pkg = tree_open_pkg(cat_ctx, atom->PF);
 	pkg->atom = atom_clone(atom);
-	pkg->name = pkg->atom->PF;
+	pkg->name = xstrdup(pkg->atom->PF);
+	pkg->repo = tctx->repo != NULL ? xstrdup(tctx->repo) : NULL;
+	if (meta != NULL) {
+		pkg->meta = xmalloc(sizeof(*pkg->meta));
+		memcpy(pkg->meta, meta, sizeof(*pkg->meta));
+		pkg->fd = 0;  /* don't try to read, we already got it */
+	} else {
+		pkg->meta = NULL;
+	}
 
 	return 0;
 }
@@ -1761,7 +1783,8 @@ tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 			snprintf(n->path, sizeof(n->path), "%s/%s/%s%s", \
 					(char *)C->ctx->path, C->name, pkg_ctx->name, \
 					C->ctx->cachetype == CACHE_EBUILD ? ".ebuild" : \
-					C->ctx->cachetype == CACHE_BINPKGS ? ".tbz2" : ""); \
+					C->ctx->cachetype == CACHE_BINPKGS ? ".tbz2" : \
+					C->ctx->cachetype == CACHE_PACKAGES ? ".tbz2" : ""); \
 			if (flags & TREE_MATCH_METADATA) \
 				n->meta = tree_pkg_read(pkg_ctx); \
 			n->next = ret; \

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     90fe066073722807b6a48f391e6500b28398830b
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Jun 14 09:30:23 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Jun 14 09:30:23 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=90fe0660

libq/tree: make tree_match_atom produce valid meta for binpkgs

copy meta appropriately in populate_cb when we deal with binpkgs
directly

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 54 ++++++++++++++++++++++++++++++++++++------------------
 1 file changed, 36 insertions(+), 18 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 5a505f2..847a343 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -546,7 +546,10 @@ tree_next_pkg(tree_cat_ctx *cat_ctx)
 	tree_ctx *ctx = cat_ctx->ctx;
 	tree_pkg_ctx *ret = NULL;
 
-	if (ctx->cachetype == CACHE_EBUILD) {
+	if (ctx->do_sort && cat_ctx->pkg_ctxs != NULL) {
+		/* bypass to use the cache if it exists */
+		ret = tree_next_pkg_int(cat_ctx);
+	} else if (ctx->cachetype == CACHE_EBUILD) {
 		char *p;
 
 		/* serve *.ebuild files each as separate pkg_ctx with name set
@@ -1600,9 +1603,6 @@ tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete)
 				}
 				pkg_ctx->atom->REPO = pkg_ctx->repo;
 			}
-
-			if (meta != NULL)
-				tree_close_meta(meta);
 		}
 
 		/* this is a bit atom territory, but since we pulled in SLOT we
@@ -1659,12 +1659,12 @@ tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms)
 static int
 tree_match_atom_cache_populate_cb(tree_pkg_ctx *ctx, void *priv)
 {
-	set *cache = priv;
-	tree_cat_ctx *cat_ctx;
-	tree_pkg_ctx *pkg;
-	tree_ctx *tctx = ctx->cat_ctx->ctx;
-	depend_atom *atom = tree_get_atom(ctx, true);
-	tree_pkg_meta *meta = tree_pkg_read(ctx);
+	tree_cat_ctx  *cat_ctx;
+	tree_pkg_ctx  *pkg;
+	set           *cache   = priv;
+	tree_ctx      *tctx    = ctx->cat_ctx->ctx;
+	depend_atom   *atom    = tree_get_atom(ctx, true);
+	tree_pkg_meta *meta    = tree_pkg_read(ctx);
 
 	(void)priv;
 
@@ -1686,10 +1686,31 @@ tree_match_atom_cache_populate_cb(tree_pkg_ctx *ctx, void *priv)
 	pkg->name = xstrdup(pkg->atom->PF);
 	pkg->repo = tctx->repo != NULL ? xstrdup(tctx->repo) : NULL;
 	if (meta != NULL) {
+		pkg->fd = -2;  /* don't try to read, we fill it in here */
 		pkg->meta = xmalloc(sizeof(*pkg->meta));
 		memcpy(pkg->meta, meta, sizeof(*pkg->meta));
-		pkg->meta->Q__data = NULL;  /* avoid free here (just to be sure) */
-		pkg->fd = -2;  /* don't try to read, we already got it */
+		if (ctx->cat_ctx->ctx->cachetype == CACHE_PACKAGES) {
+			pkg->meta->Q__data = NULL;  /* avoid free here (just to be sure) */
+		} else {  /* CACHE_BINPKG */
+			char **newdata;
+			int    elems;
+			size_t datasize = (size_t)meta->Q__eclasses_;
+
+			pkg->meta->Q__data = xmalloc(sizeof(char) * datasize);
+			memcpy(pkg->meta->Q__data, meta->Q__data, datasize);
+
+			/* re-position keys */
+			newdata = (char **)pkg->meta;
+			elems   = sizeof(tree_pkg_meta) / sizeof(char *);
+			while (elems-- > 1)  /* skip Q__data itself */
+				if (newdata[elems] != NULL)
+					newdata[elems] = pkg->meta->Q__data +
+						(newdata[elems] - meta->Q__data);
+
+			/* drop garbage used for Q__data admin in original case */
+			pkg->meta->Q__eclasses_ = NULL;
+			pkg->meta->Q__md5_      = NULL;
+		}
 	} else {
 		pkg->meta = NULL;
 	}
@@ -1728,12 +1749,9 @@ tree_match_atom(tree_ctx *ctx, const depend_atom *query, int flags)
 		else
 			cache = create_set();
 
-		if (ctx->cachetype == CACHE_PACKAGES)
-			tree_foreach_packages(ctx,
-					tree_match_atom_cache_populate_cb, cache);
-		else  /* BINPKG */
-			tree_foreach_pkg(ctx,
-					tree_match_atom_cache_populate_cb, cache, true, NULL);
+		tree_foreach_pkg(ctx,
+				tree_match_atom_cache_populate_cb, cache, true, NULL);
+
 		ctx->do_sort = true;  /* turn it back on */
 		ctx->cache.all_categories = true;
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     f3ce28b07839e478051947ae7f04bc4b798c1345
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Jun 12 15:37:46 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Jun 12 15:37:46 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=f3ce28b0

libq/tree: avoid crash in tree_close_pkg on virtuals

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 2180867..39beac8 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1221,7 +1221,7 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 			ctx->cachetype = CACHE_PACKAGES;
 			if (pkg_ctx->meta == NULL) {
 				/* hrmffff. */
-				pkg_ctx->fd = 0;
+				pkg_ctx->fd = -2;
 				pkg_ctx->meta = tree_pkg_read(pkg_ctx);
 			}
 			key = (char **)((char *)&pkg_ctx->meta->Q__data + offset);
@@ -1324,7 +1324,7 @@ tree_close_metadata(tree_metadata_xml *meta_ctx)
 void
 tree_close_pkg(tree_pkg_ctx *pkg_ctx)
 {
-	if (pkg_ctx->fd != -1)
+	if (pkg_ctx->fd >= 0)
 		close(pkg_ctx->fd);
 	if (pkg_ctx->atom != NULL)
 		atom_implode(pkg_ctx->atom);
@@ -1424,7 +1424,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 				pkg.repo = ctx->repo;
 				pkg.atom = atom;
 				pkg.cat_ctx = cat;
-				pkg.fd = 0;  /* intentional, meta has already been read */
+				pkg.fd = -2;  /* intentional, meta has already been read */
 
 				/* do call callback with pkg_atom (populate cat and pkg) */
 				ret |= callback(&pkg, priv);
@@ -1690,7 +1690,8 @@ tree_match_atom_cache_populate_cb(tree_pkg_ctx *ctx, void *priv)
 	if (meta != NULL) {
 		pkg->meta = xmalloc(sizeof(*pkg->meta));
 		memcpy(pkg->meta, meta, sizeof(*pkg->meta));
-		pkg->fd = 0;  /* don't try to read, we already got it */
+		pkg->meta->Q__data = NULL;  /* avoid free here */
+		pkg->fd = -2;  /* don't try to read, we already got it */
 	} else {
 		pkg->meta = NULL;
 	}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e8f5295c91f6e818455eac7414b17965ccbba061
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jun 13 15:05:23 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jun 13 15:05:23 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e8f5295c

libq/tree: make some declarations const

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 9 +++++----
 libq/tree.h | 7 ++++---
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index a247f66..5a505f2 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -468,7 +468,7 @@ tree_next_pkg_int(tree_cat_ctx *cat_ctx)
 {
 	tree_pkg_ctx *pkg_ctx = NULL;
 	const struct dirent *de;
-	depend_atom *qa = cat_ctx->ctx->query_atom;
+	const depend_atom *qa = cat_ctx->ctx->query_atom;
 
 	if (cat_ctx->ctx->do_sort) {
 		if (cat_ctx->pkg_ctxs == NULL) {
@@ -1349,7 +1349,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 	char pkgname[_Q_PATH_MAX];
 	size_t len;
 	int ret = 0;
-	depend_atom *query = ctx->query_atom;
+	const depend_atom *query = ctx->query_atom;
 
 	/* reused for every entry */
 	tree_cat_ctx *cat = NULL;
@@ -1510,7 +1510,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 
 int
 tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
-		bool sort, depend_atom *query)
+		bool sort, const depend_atom *query)
 {
 	tree_cat_ctx *cat_ctx;
 	tree_pkg_ctx *pkg_ctx;
@@ -1698,7 +1698,7 @@ tree_match_atom_cache_populate_cb(tree_pkg_ctx *ctx, void *priv)
 }
 
 tree_match_ctx *
-tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
+tree_match_atom(tree_ctx *ctx, const depend_atom *query, int flags)
 {
 	tree_cat_ctx *cat_ctx;
 	tree_pkg_ctx *pkg_ctx;
@@ -1779,6 +1779,7 @@ tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 			/* create a new match result */ \
 			n = xzalloc(sizeof(tree_match_ctx)); \
 			n->atom = atom; \
+			n->pkg = pkg_ctx; \
 			snprintf(n->path, sizeof(n->path), "%s/%s/%s%s", \
 					(char *)C->ctx->path, C->name, pkg_ctx->name, \
 					C->ctx->cachetype == CACHE_EBUILD ? ".ebuild" : \

diff --git a/libq/tree.h b/libq/tree.h
index 1f28205..0aa534d 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -45,7 +45,7 @@ struct tree_ctx {
 	char *repo;
 	char *pkgs;
 	size_t pkgslen;
-	depend_atom *query_atom;
+	const depend_atom *query_atom;
 	struct tree_cache {
 		char *store;
 		size_t storesize;
@@ -125,6 +125,7 @@ struct tree_metadata_xml {
  * (populated and deep copied) when set */
 struct tree_match_ctx {
 	depend_atom *atom;
+	tree_pkg_ctx *pkg;
 	tree_pkg_meta *meta;
 	char path[_Q_PATH_MAX + 48];
 	tree_match_ctx *next;
@@ -151,14 +152,14 @@ tree_metadata_xml *tree_pkg_metadata(tree_pkg_ctx *pkg_ctx);
 void tree_close_metadata(tree_metadata_xml *meta_ctx);
 void tree_close_pkg(tree_pkg_ctx *pkg_ctx);
 int tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
-		bool sort, depend_atom *query);
+		bool sort, const depend_atom *query);
 #define tree_foreach_pkg_fast(ctx, cb, priv, query) \
 	tree_foreach_pkg(ctx, cb, priv, false, query);
 #define tree_foreach_pkg_sorted(ctx, cb, priv, query) \
 	tree_foreach_pkg(ctx, cb, priv, true, query);
 set *tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms);
 depend_atom *tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete);
-tree_match_ctx *tree_match_atom(tree_ctx *t, depend_atom *q, int flags);
+tree_match_ctx *tree_match_atom(tree_ctx *t, const depend_atom *q, int flags);
 #define TREE_MATCH_FULL_ATOM  (1<<1)
 #define TREE_MATCH_METADATA   (1<<2)
 #define TREE_MATCH_LATEST     (1<<3)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     3cd1221ff6fdd8b3af243f390569b6e61bfd9e18
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Jun 12 20:03:13 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Jun 12 20:03:13 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=3cd1221f

libq/tree: fix double free/use after free scenarios

- ensure we don't reuse pointers too many so we end up freeing the wrong
  thing
- don't free atom and meta in case of cached pkg_ctx

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 39 ++++++++++++++++++++-------------------
 libq/tree.h |  5 ++++-
 2 files changed, 24 insertions(+), 20 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 39beac8..a247f66 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -162,11 +162,11 @@ tree_open_binpkg(const char *sroot, const char *spkg)
 		ret->cachetype = CACHE_BINPKGS;
 
 		fd = openat(ret->tree_fd, binpkg_packages, O_RDONLY | O_CLOEXEC);
-		if (eat_file_fd(fd, &ret->pkgs, &ret->pkgslen)) {
+		if (eat_file_fd(fd, &ret->cache.store, &ret->cache.storesize)) {
 			ret->cachetype = CACHE_PACKAGES;
-		} else if (ret->pkgs != NULL) {
-			free(ret->pkgs);
-			ret->pkgs = NULL;
+		} else if (ret->cache.store != NULL) {
+			free(ret->cache.store);
+			ret->cache.store = NULL;
 		}
 		close(fd);
 	}
@@ -1358,14 +1358,13 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 	depend_atom *atom = NULL;
 
 	/* re-read the contents, this is necessary to make it possible to
-	 * call this function multiple times
-	 * TODO: generate an internal in-memory tree when cache is enabled */
-	if (ctx->pkgs == NULL || ctx->pkgs[0] == '\0') {
+	 * call this function multiple times */
+	if (ctx->cache.store == NULL || ctx->cache.store[0] == '\0') {
 		int fd = openat(ctx->tree_fd, binpkg_packages, O_RDONLY | O_CLOEXEC);
-		if (!eat_file_fd(fd, &ctx->pkgs, &ctx->pkgslen)) {
-			if (ctx->pkgs != NULL) {
-				free(ctx->pkgs);
-				ctx->pkgs = NULL;
+		if (!eat_file_fd(fd, &ctx->cache.store, &ctx->cache.storesize)) {
+			if (ctx->cache.store != NULL) {
+				free(ctx->cache.store);
+				ctx->cache.store = NULL;
 			}
 			close(fd);
 			return 1;
@@ -1373,8 +1372,8 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 		close(fd);
 	}
 
-	p = ctx->pkgs;
-	len = strlen(ctx->pkgs);  /* sucks, need eat_file change */
+	p = ctx->cache.store;
+	len = strlen(ctx->cache.store);  /* sucks, need eat_file change */
 
 	memset(&meta, 0, sizeof(meta));
 
@@ -1396,9 +1395,8 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 
 				memset(&pkg, 0, sizeof(pkg));
 
-				/* store meta ptr in repo->pkgs, such that get_pkg_meta
+				/* store meta ptr in ctx->pkgs, such that get_pkg_meta
 				 * can grab it from there (for free) */
-				c = ctx->pkgs;
 				ctx->pkgs = (char *)&meta;
 
 				if (cat == NULL || strcmp(cat->name, atom->CATEGORY) != 0)
@@ -1429,7 +1427,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 				/* do call callback with pkg_atom (populate cat and pkg) */
 				ret |= callback(&pkg, priv);
 
-				ctx->pkgs = c;
+				ctx->pkgs = NULL;
 				if (atom != (depend_atom *)cat->pkg_ctxs)
 					atom_implode(atom);
 			}
@@ -1505,7 +1503,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 	/* ensure we don't free a garbage pointer */
 	ctx->repo = NULL;
 	ctx->do_sort = false;
-	ctx->pkgs[0] = '\0';
+	ctx->cache.store[0] = '\0';
 
 	return ret;
 }
@@ -1690,7 +1688,7 @@ tree_match_atom_cache_populate_cb(tree_pkg_ctx *ctx, void *priv)
 	if (meta != NULL) {
 		pkg->meta = xmalloc(sizeof(*pkg->meta));
 		memcpy(pkg->meta, meta, sizeof(*pkg->meta));
-		pkg->meta->Q__data = NULL;  /* avoid free here */
+		pkg->meta->Q__data = NULL;  /* avoid free here (just to be sure) */
 		pkg->fd = -2;  /* don't try to read, we already got it */
 	} else {
 		pkg->meta = NULL;
@@ -1788,6 +1786,9 @@ tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 					C->ctx->cachetype == CACHE_PACKAGES ? ".tbz2" : ""); \
 			if (flags & TREE_MATCH_METADATA) \
 				n->meta = tree_pkg_read(pkg_ctx); \
+			if (C->ctx->cachetype == CACHE_BINPKGS || \
+					C->ctx->cachetype == CACHE_PACKAGES) \
+				n->free_atom = n->free_meta = 0; \
 			n->next = ret; \
 			ret = n; \
 			lastpn = atom->PN; \
@@ -1825,7 +1826,7 @@ tree_match_close(tree_match_ctx *match)
 		w = match->next;
 		if (match->free_atom)
 			atom_implode(match->atom);
-		if (match->meta != NULL)
+		if (match->free_meta && match->meta != NULL)
 			tree_close_meta(match->meta);
 		free(match);
 	}

diff --git a/libq/tree.h b/libq/tree.h
index 8741bad..1f28205 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -47,6 +47,8 @@ struct tree_ctx {
 	size_t pkgslen;
 	depend_atom *query_atom;
 	struct tree_cache {
+		char *store;
+		size_t storesize;
 		set *categories;
 		bool all_categories:1;
 	} cache;
@@ -126,7 +128,8 @@ struct tree_match_ctx {
 	tree_pkg_meta *meta;
 	char path[_Q_PATH_MAX + 48];
 	tree_match_ctx *next;
-	int free_atom;
+	char free_atom:1;
+	char free_meta:1;
 };
 
 /* foreach pkg callback function signature */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     4fd296d65c76ceb98f69ff392c2ae0f31bd18d54
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Jun  1 19:38:52 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue Jun  1 19:38:52 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=4fd296d6

libq/tree: fix empty cat_ctx and path when using binpkgs

- ensure cat_ctx is set for each pkg coming from Packages file
- set path in each tree match

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index c00f251..358b1f2 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1380,7 +1380,6 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 						 * generate a dummy cat */
 						cat = tree_open_cat(ctx, ".");
 					}
-					pkg.cat_ctx = cat;
 					cat->pkg_ctxs = (tree_pkg_ctx **)atom;  /* for name */
 				}
 				pkgnamelen = snprintf(pkgname, sizeof(pkgname),
@@ -1390,6 +1389,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 				pkg.slot = meta.Q_SLOT == NULL ? (char *)"0" : meta.Q_SLOT;
 				pkg.repo = ctx->repo;
 				pkg.atom = atom;
+				pkg.cat_ctx = cat;
 				pkg.fd = 0;  /* intentional, meta has already been read */
 
 				/* do call callback with pkg_atom (populate cat and pkg) */
@@ -1650,6 +1650,8 @@ tree_match_atom_packages_cb(tree_pkg_ctx *ctx, void *priv)
 	n = xzalloc(sizeof(tree_match_ctx));
 	n->free_atom = true;
 	n->atom = atom_clone(a);
+	snprintf(n->path, sizeof(n->path), "%s/%s/%s.tbz2",
+			(char *)ctx->cat_ctx->ctx->path, ctx->cat_ctx->name, ctx->name);
 	if (rctx->flags & TREE_MATCH_METADATA) {
 		n->meta = xmalloc(sizeof(*n->meta));
 		/* for Packages, all pointers to meta here are to the in memory
@@ -1686,6 +1688,8 @@ tree_match_atom_binpkg_cb(tree_pkg_ctx *ctx, void *priv)
 	n = xzalloc(sizeof(tree_match_ctx));
 	n->free_atom = true;
 	n->atom = atom_clone(a);
+	snprintf(n->path, sizeof(n->path), "%s/%s/%s.tbz2",
+			(char *)ctx->cat_ctx->ctx->path, ctx->cat_ctx->name, ctx->name);
 	if (rctx->flags & TREE_MATCH_METADATA)
 		n->meta = tree_pkg_read(ctx);
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     6e42f76862860f19674fe1f87d9a7087acfa742e
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun May 23 10:54:00 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun May 23 10:54:00 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=6e42f768

libq/tree: silence warning about unknown keys in md5 cache format

Bug: https://bugs.gentoo.org/791571
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index f66059b..c00f251 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -785,8 +785,8 @@ tree_read_file_md5(tree_pkg_ctx *pkg_ctx)
 		assign_var(EPREFIX);
 		assign_var(_eclasses_);
 		assign_var(_md5_);
-		warn("Cache file for '%s/%s' has unknown key %s",
-				pkg_ctx->cat_ctx->name, pkg_ctx->name, keyptr);
+		IF_DEBUG(warn("Cache file for '%s/%s' has unknown key %s",
+					pkg_ctx->cat_ctx->name, pkg_ctx->name, keyptr));
 	}
 #undef assign_var
 #undef assign_var_cmp

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     09e2ac80fc030013031135bc3ffad19360920d6f
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 10 09:11:25 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 10 09:11:25 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=09e2ac80

libq/colors: add color_clear()

allow calling color_clear()/color_remap() repeatedly to disable/enable
colour escapes (this is necessary for the delayed argument handling by
the applets)

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/colors.c | 92 +++++++++++++++++++++++++++++++++++++++--------------------
 libq/colors.h |  3 +-
 2 files changed, 63 insertions(+), 32 deletions(-)

diff --git a/libq/colors.c b/libq/colors.c
index 9a48e2e..6f3d7f1 100644
--- a/libq/colors.c
+++ b/libq/colors.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2021 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
@@ -17,24 +17,22 @@
 #else
 # define _MAKE_COLOR(c,b) "\e[" c ";" b "m"
 #endif
-const char *BOLD = _MAKE_COLOR("00", "01");
-const char *NORM = _MAKE_COLOR("00", "00");
-const char *BLUE = _MAKE_COLOR("36", "01");
-const char *DKBLUE = _MAKE_COLOR("34", "01");
-const char *CYAN = _MAKE_COLOR("00", "36");
-const char *GREEN = _MAKE_COLOR("32", "01");
-const char *DKGREEN = _MAKE_COLOR("00", "32");
-const char *MAGENTA = _MAKE_COLOR("00", "35");
-const char *RED = _MAKE_COLOR("31", "01");
-const char *YELLOW = _MAKE_COLOR("33", "01");
-const char *BRYELLOW = _MAKE_COLOR("01", "33");
-const char *WHITE = _MAKE_COLOR("01", "38");
+const char *NORM;
+const char *BLUE;
+const char *BOLD;
+const char *BRYELLOW;
+const char *CYAN;
+const char *DKBLUE;
+const char *DKGREEN;
+const char *GREEN;
+const char *MAGENTA;
+const char *RED;
+const char *WHITE;
+const char *YELLOW;
 
 static const char *COLOR_MAP = CONFIG_EPREFIX "etc/portage/color.map";
 
-#define COLOR _MAKE_COLOR
 #define CPAIR_VALUE_LEN 16
-
 typedef struct {
 	const char *name;
 	char value[CPAIR_VALUE_LEN];
@@ -43,22 +41,22 @@ typedef struct {
 
 #define X2(X) X, X
 static cpairtype color_pairs[] = {
-	{"blue",      X2(COLOR("34", "01")) },
-	{"brown",     X2(COLOR("00", "33")) },
-	{"darkblue",  X2(COLOR("00", "34")) },
-	{"darkgreen", X2(COLOR("00", "32")) },
-	{"darkred",   X2(COLOR("00", "31")) },
-	{"faint",     X2(COLOR("00", "02")) },
-	{"fuchsia",   X2(COLOR("35", "01")) },
-	{"green",     X2(COLOR("32", "01")) },
-	{"purple",    X2(COLOR("00", "35")) },
-	{"red",       X2(COLOR("31", "01")) },
-	{"teal",      X2(COLOR("00", "36")) },
-	{"turquoise", X2(COLOR("36", "01")) },
-	{"yellow",    X2(COLOR("01", "33")) },
-	{"white",     X2(COLOR("01", "38")) },
-	{"lightgray", X2(COLOR("00", "37")) },
-	{"eol",       X2(COLOR("00", "00")) },
+	{"blue",      X2(_MAKE_COLOR("34", "01")) },
+	{"brown",     X2(_MAKE_COLOR("00", "33")) },
+	{"darkblue",  X2(_MAKE_COLOR("00", "34")) },
+	{"darkgreen", X2(_MAKE_COLOR("00", "32")) },
+	{"darkred",   X2(_MAKE_COLOR("00", "31")) },
+	{"faint",     X2(_MAKE_COLOR("00", "02")) },
+	{"fuchsia",   X2(_MAKE_COLOR("35", "01")) },
+	{"green",     X2(_MAKE_COLOR("32", "01")) },
+	{"lightgray", X2(_MAKE_COLOR("00", "37")) },
+	{"purple",    X2(_MAKE_COLOR("00", "35")) },
+	{"red",       X2(_MAKE_COLOR("31", "01")) },
+	{"teal",      X2(_MAKE_COLOR("00", "36")) },
+	{"turquoise", X2(_MAKE_COLOR("36", "01")) },
+	{"white",     X2(_MAKE_COLOR("01", "38")) },
+	{"yellow",    X2(_MAKE_COLOR("01", "33")) },
+	{"eol",       X2(_MAKE_COLOR("00", "00")) },
 };
 #undef X2
 
@@ -73,6 +71,21 @@ color_remap(void)
 	char *p;
 	unsigned int lineno = 0;
 
+	/* set q's defaults, if there's no colormap, or the file is empty,
+	 * or it doesn't match things, we at least got some defaults */
+	NORM     = _MAKE_COLOR("00", "00");
+	BLUE     = _MAKE_COLOR("36", "01");
+	BOLD     = _MAKE_COLOR("00", "01");
+	BRYELLOW = _MAKE_COLOR("01", "33");
+	CYAN     = _MAKE_COLOR("00", "36");
+	DKBLUE   = _MAKE_COLOR("34", "01");
+	DKGREEN  = _MAKE_COLOR("00", "32");
+	GREEN    = _MAKE_COLOR("32", "01");
+	MAGENTA  = _MAKE_COLOR("00", "35");
+	RED      = _MAKE_COLOR("31", "01");
+	WHITE    = _MAKE_COLOR("01", "38");
+	YELLOW   = _MAKE_COLOR("33", "01");
+
 	if ((fp = fopen(COLOR_MAP, "r")) == NULL)
 		return;
 
@@ -141,3 +154,20 @@ color_remap(void)
 			CYAN = color_pairs[i].value;
 	}
 }
+
+void
+color_clear(void)
+{
+	NORM     = "";
+	BLUE     = "";
+	BOLD     = "";
+	BRYELLOW = "";
+	CYAN     = "";
+	DKBLUE   = "";
+	DKGREEN  = "";
+	GREEN    = "";
+	MAGENTA  = "";
+	RED      = "";
+	WHITE    = "";
+	YELLOW   = "";
+}

diff --git a/libq/colors.h b/libq/colors.h
index 9947bc1..2c3de9b 100644
--- a/libq/colors.h
+++ b/libq/colors.h
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2021 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2019-     Fabian Groffen  - <grobian@gentoo.org>
@@ -22,5 +22,6 @@ extern const char *BRYELLOW;
 extern const char *WHITE;
 
 void color_remap(void);
+void color_clear(void);
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     7bbb7d1237d4fc6380c29059c7b0531d2adf0ee4
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Apr 29 15:03:18 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Apr 29 15:03:18 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=7bbb7d12

libq/tree: return all pkgs found in Packages file, whether or not existant

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index e692ee9..f66059b 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1374,18 +1374,13 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 						cat->pkg_ctxs = NULL;
 						tree_close_cat(cat);
 					}
-					pkg.cat_ctx = cat = tree_open_cat(ctx, atom->CATEGORY);
+					cat = tree_open_cat(ctx, atom->CATEGORY);
 					if (cat == NULL) {
-						/* probably dir doesn't exist or something, skip
-						 * this one */
-						memset(&meta, 0, sizeof(meta));
-						if (len > 0) {  /* hop over \n */
-							p++;
-							len--;
-						}
-						ctx->pkgs = c;
-						continue;
+						/* probably dir doesn't exist or something,
+						 * generate a dummy cat */
+						cat = tree_open_cat(ctx, ".");
 					}
+					pkg.cat_ctx = cat;
 					cat->pkg_ctxs = (tree_pkg_ctx **)atom;  /* for name */
 				}
 				pkgnamelen = snprintf(pkgname, sizeof(pkgname),

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     76007ed6b28a5bc8c9b108bdf141fd31357fde6c
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Apr 29 13:46:37 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Apr 29 13:46:37 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=76007ed6

tree: avoid crash at close when a binpkg is skipped

this fixes up the previous commit 09b8177

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libq/tree.c b/libq/tree.c
index 04bf468..e692ee9 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1383,6 +1383,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 							p++;
 							len--;
 						}
+						ctx->pkgs = c;
 						continue;
 					}
 					cat->pkg_ctxs = (tree_pkg_ctx **)atom;  /* for name */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     09b817763d3b03d9ccee92ddd4fa13edb82d53a9
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Apr 29 13:23:27 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Apr 29 13:23:27 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=09b81776

tree: avoid crash in tree_foreach_packages when category doesn't exist

This can be observed when reading a Packages file that refers to binpkgs
that do not exist.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index a947735..04bf468 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1375,6 +1375,16 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 						tree_close_cat(cat);
 					}
 					pkg.cat_ctx = cat = tree_open_cat(ctx, atom->CATEGORY);
+					if (cat == NULL) {
+						/* probably dir doesn't exist or something, skip
+						 * this one */
+						memset(&meta, 0, sizeof(meta));
+						if (len > 0) {  /* hop over \n */
+							p++;
+							len--;
+						}
+						continue;
+					}
 					cat->pkg_ctxs = (tree_pkg_ctx **)atom;  /* for name */
 				}
 				pkgnamelen = snprintf(pkgname, sizeof(pkgname),

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     2cff8ef68b128ffe8f2074a6a8a58795ab7f06b4
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Mar 13 12:42:58 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Mar 13 12:42:58 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=2cff8ef6

libq/tree: fix tree_match_atom when returning newest match

we were not evaluating any versions after the first we've seen for each
package, disallowing us to match any but the most recent version

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index 9b06720..a947735 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1758,8 +1758,8 @@ tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 				n->meta = tree_pkg_read(pkg_ctx); \
 			n->next = ret; \
 			ret = n; \
+			lastpn = atom->PN; \
 		} \
-		lastpn = atom->PN; \
 		if (flags & TREE_MATCH_FIRST && ret != NULL) \
 			break; \
 	} \

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     a94a9e688526d776c808fee96e9a3b9e1733764c
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 20 12:04:32 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Feb 20 12:04:32 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=a94a9e68

libq/tree: allow tree_match_atom to skip over acct-* categories

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 4 ++++
 libq/tree.h | 7 ++++---
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index df52171..9b06720 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1736,6 +1736,10 @@ tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 		if (!(flags & TREE_MATCH_VIRTUAL || \
 				strcmp(atom->CATEGORY, "virtual") != 0)) \
 			continue; \
+		/* skip acct-* package as requested */ \
+		if (!(flags & TREE_MATCH_ACCT || \
+				strncmp(atom->CATEGORY, "acct-", sizeof("acct-") - 1) != 0)) \
+			continue; \
 		/* see if this atom matches the query */ \
 		if (atom_compare(atom, query) == EQUAL) { \
 			tree_match_ctx *n; \

diff --git a/libq/tree.h b/libq/tree.h
index 53fee67..5a12c92 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -158,9 +158,10 @@ tree_match_ctx *tree_match_atom(tree_ctx *t, depend_atom *q, int flags);
 #define TREE_MATCH_FULL_ATOM  (1<<1)
 #define TREE_MATCH_METADATA   (1<<2)
 #define TREE_MATCH_LATEST     (1<<3)
-#define TREE_MATCH_VIRTUAL    (1<<4)
-#define TREE_MATCH_FIRST      (1<<5)
-#define TREE_MATCH_DEFAULT     TREE_MATCH_VIRTUAL
+#define TREE_MATCH_FIRST      (1<<4)
+#define TREE_MATCH_VIRTUAL    (1<<5)
+#define TREE_MATCH_ACCT       (1<<6)
+#define TREE_MATCH_DEFAULT    (TREE_MATCH_VIRTUAL | TREE_MATCH_ACCT)
 void tree_match_close(tree_match_ctx *t);
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     90de2c63f3d0d7e546265b02227b96b01dba24f7
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 20 10:54:05 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Feb 20 10:54:05 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=90de2c63

libq/tree: add ability to match latest version per package in tree_match_atom

Allow retrieving the latest version per package matching the input atom.
E.g. a search for dash would return both app-shells/dash as well as
app-emacs/dash using LATEST, while FIRST would only return the app-emacs
one (or whichever came first while traversing).

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 42 ++++++++++++++++++++++++++----------------
 libq/tree.h |  9 +++++----
 2 files changed, 31 insertions(+), 20 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 293d7f7..df52171 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1728,26 +1728,36 @@ tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 
 #define search_cat(C) \
 { \
+	char *lastpn = NULL; \
 	while ((pkg_ctx = tree_next_pkg(C)) != NULL) { \
 		atom = tree_get_atom(pkg_ctx, \
 				query->SLOT != NULL || flags & TREE_MATCH_FULL_ATOM); \
-		if (flags & TREE_MATCH_VIRTUAL || \
-				strcmp(atom->CATEGORY, "virtual") != 0) { \
-			if (atom_compare(atom, query) == EQUAL) { \
-				tree_match_ctx *n = xzalloc(sizeof(tree_match_ctx)); \
-				n->atom = atom; \
-				snprintf(n->path, sizeof(n->path), "%s/%s/%s%s", \
-						(char *)C->ctx->path, C->name, pkg_ctx->name, \
-						C->ctx->cachetype == CACHE_EBUILD ? ".ebuild" : \
-						C->ctx->cachetype == CACHE_BINPKGS ? ".tbz2" : ""); \
-				if (flags & TREE_MATCH_METADATA) \
-					n->meta = tree_pkg_read(pkg_ctx); \
-				n->next = ret; \
-				ret = n; \
-			} \
-			if (flags & TREE_MATCH_FIRST && ret != NULL) \
-				break; \
+		/* skip virtual/ package as requested */ \
+		if (!(flags & TREE_MATCH_VIRTUAL || \
+				strcmp(atom->CATEGORY, "virtual") != 0)) \
+			continue; \
+		/* see if this atom matches the query */ \
+		if (atom_compare(atom, query) == EQUAL) { \
+			tree_match_ctx *n; \
+			/* skip over additional versions for match latest */ \
+			if (flags & TREE_MATCH_LATEST && lastpn != NULL && \
+					strcmp(lastpn, atom->PN) == 0) \
+				continue; \
+			/* create a new match result */ \
+			n = xzalloc(sizeof(tree_match_ctx)); \
+			n->atom = atom; \
+			snprintf(n->path, sizeof(n->path), "%s/%s/%s%s", \
+					(char *)C->ctx->path, C->name, pkg_ctx->name, \
+					C->ctx->cachetype == CACHE_EBUILD ? ".ebuild" : \
+					C->ctx->cachetype == CACHE_BINPKGS ? ".tbz2" : ""); \
+			if (flags & TREE_MATCH_METADATA) \
+				n->meta = tree_pkg_read(pkg_ctx); \
+			n->next = ret; \
+			ret = n; \
 		} \
+		lastpn = atom->PN; \
+		if (flags & TREE_MATCH_FIRST && ret != NULL) \
+			break; \
 	} \
 	C->pkg_cur = 0;  /* reset to allow another traversal */ \
 }

diff --git a/libq/tree.h b/libq/tree.h
index f756fd4..53fee67 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -155,10 +155,11 @@ int tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 set *tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms);
 depend_atom *tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete);
 tree_match_ctx *tree_match_atom(tree_ctx *t, depend_atom *q, int flags);
-#define TREE_MATCH_FULL_ATOM   1<<1
-#define TREE_MATCH_METADATA    1<<2
-#define TREE_MATCH_FIRST       1<<3
-#define TREE_MATCH_VIRTUAL     1<<4
+#define TREE_MATCH_FULL_ATOM  (1<<1)
+#define TREE_MATCH_METADATA   (1<<2)
+#define TREE_MATCH_LATEST     (1<<3)
+#define TREE_MATCH_VIRTUAL    (1<<4)
+#define TREE_MATCH_FIRST      (1<<5)
 #define TREE_MATCH_DEFAULT     TREE_MATCH_VIRTUAL
 void tree_match_close(tree_match_ctx *t);
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     16ff9817f7d68d37e39302dac6f632c306cdce4f
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Feb 17 20:14:01 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Feb 17 20:14:01 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=16ff9817

libq/tree: make tree_match_atom return path for each match

Along with some changes to make sure returned atoms remain valid (e.g.
not getting an expired pointer for category name), build a patch to the
returned objects, that should allow opening the ebuild or something.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 41 +++++++++++++++++++++++++++++++++--------
 libq/tree.h |  5 ++++-
 2 files changed, 37 insertions(+), 9 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index e41a0cb..293d7f7 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2020 Gentoo Foundation
+ * Copyright 2005-2021 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
@@ -55,6 +55,10 @@ tree_open_int(const char *sroot, const char *tdir, bool quiet)
 		goto cp_error;
 	}
 
+	if (sroot[0] == '/' && sroot[1] == '\0')
+		sroot = "";
+	snprintf(ctx->path, sizeof(ctx->path), "%s/%s", sroot, tdir);
+
 	ctx->dir = fdopendir(ctx->tree_fd);
 	if (ctx->dir == NULL)
 		goto cv_error;
@@ -128,6 +132,25 @@ tree_open_vdb(const char *sroot, const char *svdb)
 	return ret;
 }
 
+tree_ctx *
+tree_open_ebuild(const char *sroot, const char *portdir)
+{
+	tree_ctx *ret = tree_open_int(sroot, portdir, true);
+	if (ret != NULL) {
+		char buf[_Q_PATH_MAX];
+		char *repo = NULL;
+		size_t repolen = 0;
+
+		snprintf(buf, sizeof(buf), "%s%s/%s", sroot, portdir, portrepo_name);
+		if (eat_file(buf, &repo, &repolen)) {
+			(void)rmspace(repo);
+			ret->repo = repo;
+		}
+		ret->cachetype = CACHE_EBUILD;
+	}
+	return ret;
+}
+
 static const char binpkg_packages[]  = "Packages";
 tree_ctx *
 tree_open_binpkg(const char *sroot, const char *spkg)
@@ -1491,10 +1514,10 @@ depend_atom *
 tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete)
 {
 	if (pkg_ctx->atom == NULL) {
-		pkg_ctx->atom = atom_explode(pkg_ctx->name);
+		pkg_ctx->atom =
+			atom_explode_cat(pkg_ctx->name, (char *)pkg_ctx->cat_ctx->name);
 		if (pkg_ctx->atom == NULL)
 			return NULL;
-		pkg_ctx->atom->CATEGORY = (char *)pkg_ctx->cat_ctx->name;
 	}
 
 	if (complete) {
@@ -1708,14 +1731,15 @@ tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 	while ((pkg_ctx = tree_next_pkg(C)) != NULL) { \
 		atom = tree_get_atom(pkg_ctx, \
 				query->SLOT != NULL || flags & TREE_MATCH_FULL_ATOM); \
-fprintf(stderr, "fbg: %s\n", atom_to_string(atom)); \
 		if (flags & TREE_MATCH_VIRTUAL || \
-				strcmp(atom->CATEGORY, "virtual") != 0) \
+				strcmp(atom->CATEGORY, "virtual") != 0) { \
 			if (atom_compare(atom, query) == EQUAL) { \
-				tree_match_ctx *n; \
-				n = xzalloc(sizeof(tree_match_ctx)); \
-				n->free_atom = false; \
+				tree_match_ctx *n = xzalloc(sizeof(tree_match_ctx)); \
 				n->atom = atom; \
+				snprintf(n->path, sizeof(n->path), "%s/%s/%s%s", \
+						(char *)C->ctx->path, C->name, pkg_ctx->name, \
+						C->ctx->cachetype == CACHE_EBUILD ? ".ebuild" : \
+						C->ctx->cachetype == CACHE_BINPKGS ? ".tbz2" : ""); \
 				if (flags & TREE_MATCH_METADATA) \
 					n->meta = tree_pkg_read(pkg_ctx); \
 				n->next = ret; \
@@ -1723,6 +1747,7 @@ fprintf(stderr, "fbg: %s\n", atom_to_string(atom)); \
 			} \
 			if (flags & TREE_MATCH_FIRST && ret != NULL) \
 				break; \
+		} \
 	} \
 	C->pkg_cur = 0;  /* reset to allow another traversal */ \
 }

diff --git a/libq/tree.h b/libq/tree.h
index 900b998..f756fd4 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2020 Gentoo Foundation
+ * Copyright 2005-2021 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  */
 
@@ -24,6 +24,7 @@ typedef struct tree_match_ctx    tree_match_ctx;
 struct tree_ctx {
 	int portroot_fd;
 	int tree_fd;
+	char path[_Q_PATH_MAX];
 	DIR *dir;
 	struct dirent **cat_de;
 	size_t cat_cnt;
@@ -122,6 +123,7 @@ struct tree_metadata_xml {
 struct tree_match_ctx {
 	depend_atom *atom;
 	tree_pkg_meta *meta;
+	char path[_Q_PATH_MAX + 48];
 	tree_match_ctx *next;
 	int free_atom;
 };
@@ -131,6 +133,7 @@ typedef int (tree_pkg_cb)(tree_pkg_ctx *, void *priv);
 
 tree_ctx *tree_open(const char *sroot, const char *portdir);
 tree_ctx *tree_open_vdb(const char *sroot, const char *svdb);
+tree_ctx *tree_open_ebuild(const char *sroot, const char *portdir);
 tree_ctx *tree_open_binpkg(const char *sroot, const char *spkg);
 void tree_close(tree_ctx *ctx);
 tree_cat_ctx *tree_open_cat(tree_ctx *ctx, const char *name);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e3c6591032d6a4589ae98d6abb0f59d3325bc11e
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Feb 17 20:00:28 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Feb 17 20:00:28 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e3c65910

libq/atom: adapt atom_explode to take an optional category

Sometimes we don't have the category available in the to be parsed
string, but in a separate string.  Make it relatively cheap to use this
category, compared to having to allocate and format a string to make
atom_explode pick it up.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 21 ++++++++++++++-------
 libq/atom.h |  5 +++--
 2 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index acf28be..f4c7c1e 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2020 Gentoo Foundation
+ * Copyright 2005-2021 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
@@ -43,7 +43,7 @@ const char * const booga[] = {"!!!", "!=", "==", ">", "<"};
  * for a definition of which variable contains what, see:
  * https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-10800011 */
 depend_atom *
-atom_explode(const char *atom)
+atom_explode_cat(const char *atom, const char *cat)
 {
 	depend_atom *ret;
 	char *ptr;
@@ -70,14 +70,14 @@ atom_explode(const char *atom)
 	 * We allocate memory for atom struct, one string for CAT/PF + PVR,
 	 * another to cover PN and a final one for P + PV. */
 	slen = strlen(atom) + 1;
-	len = sizeof(*ret) + (slen * 3);
-	ret = xmalloc(len);
+	len = cat != NULL ? strlen(cat) + 1 : 0;
+	ret = xmalloc(sizeof(*ret) + (slen * 3) + len);
 	memset(ret, '\0', sizeof(*ret));
 
 	/* assign pointers to the three storage containers */
-	ret->CATEGORY = (char *)ret + sizeof(*ret);     /* CAT PF PVR */
-	ret->P        = ret->CATEGORY + slen;           /* P   PV     */
-	ret->PN       = ret->P + slen;                  /* PN         */
+	ret->CATEGORY = (char *)ret + sizeof(*ret) + len;     /* CAT PF PVR */
+	ret->P        = ret->CATEGORY + slen;                 /* P   PV     */
+	ret->PN       = ret->P + slen;                        /* PN         */
 
 	/* check for blocker operators */
 	ret->blocker = ATOM_BL_NONE;
@@ -245,6 +245,13 @@ atom_explode(const char *atom)
 		ret->CATEGORY = NULL;
 	}
 
+	/* inject separate CATEGORY when given, this will override any found
+	 * CATEGORY, which is what it could be used for too */
+	if (cat != NULL) {
+		ret->CATEGORY = (char *)ret + sizeof(*ret);
+		memcpy(ret->CATEGORY, cat, len);
+	}
+
 	if (ret->PF == NULL) {
 		/* atom has no name, this is it */
 		ret->P = NULL;

diff --git a/libq/atom.h b/libq/atom.h
index aff4548..ead9154 100644
--- a/libq/atom.h
+++ b/libq/atom.h
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2021 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
@@ -96,7 +96,8 @@ typedef enum {
 	OLDER
 } atom_equality;
 
-depend_atom *atom_explode(const char *atom);
+depend_atom *atom_explode_cat(const char *atom, const char *cat);
+#define atom_explode(A) atom_explode_cat(A, NULL)
 depend_atom *atom_clone(depend_atom *atom);
 void atom_implode(depend_atom *atom);
 atom_equality atom_compare(const depend_atom *a1, const depend_atom *a2);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     6480394a50b1be7632ce414ca8d956484eac51f2
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Jan 15 20:03:51 2021 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Jan 15 20:03:51 2021 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=6480394a

libq/hash: close fd once in hash_multiple_file_fd

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/hash.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/libq/hash.c b/libq/hash.c
index 88c4b56..b5aec46 100644
--- a/libq/hash.c
+++ b/libq/hash.c
@@ -147,6 +147,7 @@ hash_multiple_file_fd(
 	(void)blak2b;
 #endif
 
+	*flen = 0;
 	if ((f = fdopen(fd, "r")) == NULL)
 		return -1;
 
@@ -164,7 +165,6 @@ hash_multiple_file_fd(
 	blake2b_init(&bl2b, BLAKE2B_OUTBYTES);
 #endif
 
-	*flen = 0;
 	while ((len = fread(data, 1, sizeof(data), f)) > 0) {
 		*flen += len;
 #pragma omp parallel sections
@@ -323,7 +323,9 @@ hash_multiple_file_at_cb(
 	ret = hash_multiple_file_fd(fd, md5, sha1, sha256, sha512,
 			whrlpl, blak2b, flen, hashes);
 
-	close(fd);
+	if (ret != 0)
+		close(fd);
+
 	return ret;
 }
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     1b42d9bd178a8fddf8e31615ac37ce3bb9d94d16
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Jun 27 09:34:49 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Jun 27 09:34:49 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=1b42d9bd

libq/tree: ensure tree_foreach_pkg_fast can be run multiple times too

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index ad8db47..e41a0cb 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1477,6 +1477,8 @@ tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 	/* allow foreach to be called again on the same open tree */
 	if (ctx->do_sort)
 		scandir_free(ctx->cat_de, ctx->cat_cnt);
+	else
+		rewinddir(ctx->dir);
 	ctx->cat_de = NULL;
 	ctx->cat_cur = 0;
 	ctx->cat_cnt = 0;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     4fbeb8ccca1b93da4987b0d58aba24c0901e898d
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jun  7 10:39:40 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jun  7 10:39:40 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=4fbeb8cc

libq/tree: fix metadata cache after realloc

A realloc caused a relocation of pointers, which also changed the value
of Q__md5_ and Q__eclasses_ which are secretly used to store len and
pos, so make sure we reset them to correct values after a relocation
takes place.

While at it, reduce the allocation requirements by only considering the
really needed space (over-allocate less here).

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index d8d238c..ad8db47 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -957,9 +957,8 @@ tree_read_file_binpkg_xpak_cb(
 
 	if (len - pos < (size_t)(data_len + 1)) {
 		char *old_data = m->Q__data;
-		len += (((data_len + 1) / BUFSIZ) + 1) * BUFSIZ;
+		len += (((data_len + 1 - (len - pos)) / BUFSIZ) + 1) * BUFSIZ;
 		m->Q__data = xrealloc(m->Q__data, len);
-		m->Q__md5_ = (char *)len;
 
 		/* re-position existing keys */
 		if (old_data != NULL && m->Q__data != old_data) {
@@ -969,6 +968,10 @@ tree_read_file_binpkg_xpak_cb(
 				if (newdata[elems] != NULL)
 					newdata[elems] = m->Q__data + (newdata[elems] - old_data);
 		}
+
+		/* set after repositioning! */
+		m->Q__md5_ = (char *)len;
+		m->Q__eclasses_ = (char *)pos;
 	}
 
 	*key = m->Q__data + pos;
@@ -997,9 +1000,8 @@ tree_read_file_binpkg(tree_pkg_ctx *pkg_ctx)
 
 		if (len - pos < needlen) {
 			char *old_data = m->Q__data;
-			len += ((needlen / BUFSIZ) + 1) * BUFSIZ;
+			len += (((needlen - (len - pos)) / BUFSIZ) + 1) * BUFSIZ;
 			m->Q__data = xrealloc(m->Q__data, len);
-			m->Q__md5_ = (char *)len;
 
 			/* re-position existing keys */
 			if (old_data != NULL && m->Q__data != old_data) {
@@ -1010,6 +1012,10 @@ tree_read_file_binpkg(tree_pkg_ctx *pkg_ctx)
 						newdata[elems] =
 							m->Q__data + (newdata[elems] - old_data);
 			}
+
+			/* set after repositioning! */
+			m->Q__md5_ = (char *)len;
+			m->Q__eclasses_ = (char *)pos;
 		}
 
 		m->Q_SHA1 = m->Q__data + pos;
@@ -1112,9 +1118,8 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 			/* TODO: this is an exact copy from tree_read_file_binpkg_xpak_cb */
 			if (len - pos < (size_t)(s.st_size + 1)) {
 				p = m->Q__data;
-				len += (((s.st_size + 1) / BUFSIZ) + 1) * BUFSIZ;
+				len += (((s.st_size + 1 - (len - pos)) / BUFSIZ) + 1) * BUFSIZ;
 				m->Q__data = xrealloc(m->Q__data, len);
-				m->Q__md5_ = (char *)len;
 
 				/* re-position existing keys */
 				if (p != NULL && m->Q__data != p) {
@@ -1124,6 +1129,10 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 						if (newdata[elems] != NULL)
 							newdata[elems] = m->Q__data + (newdata[elems] - p);
 				}
+
+				/* set after repositioning! */
+				m->Q__md5_ = (char *)len;
+				m->Q__eclasses_ = (char *)pos;
 			}
 
 			p = *key = m->Q__data + pos;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     7b8a0e92eaddea4e699b0b551e2bf7ce6b8aecb1
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 18:19:15 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 18:19:15 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=7b8a0e92

libq/tree: properly free meta in tree_match_close

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index eb5f324..d8d238c 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1744,7 +1744,7 @@ tree_match_close(tree_match_ctx *match)
 		if (match->free_atom)
 			atom_implode(match->atom);
 		if (match->meta != NULL)
-			free(match->meta);
+			tree_close_meta(match->meta);
 		free(match);
 	}
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     598110a8c77b8d6995b9f34e2455742b692ff8f1
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 18:01:49 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 18:01:49 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=598110a8

libq/hash: avoid invalid (double) fclose

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/hash.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/libq/hash.c b/libq/hash.c
index a51ea6a..88c4b56 100644
--- a/libq/hash.c
+++ b/libq/hash.c
@@ -291,7 +291,6 @@ hash_multiple_file_fd(
 #endif
 	}
 
-	fclose(f);
 	return 0;
 }
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     74aeefacd106fcf87e755f2d1a53b0c7bc9f2e10
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 11:19:38 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 11:19:38 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=74aeefac

libq/tree: fix Coverity 210534

tree_close_pkg can be called from tree_open_pkg, which will then free
the pointer de->d_name, which should only be freed by scandir_free.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libq/tree.c b/libq/tree.c
index d313f3b..e4c4eb7 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -399,6 +399,7 @@ tree_open_pkg(tree_cat_ctx *cat_ctx, const char *name)
 	if (cat_ctx->ctx->query_atom != NULL) {
 		(void)tree_get_atom(pkg_ctx, cat_ctx->ctx->query_atom->SLOT != NULL);
 		if (atom_compare(pkg_ctx->atom, cat_ctx->ctx->query_atom) != EQUAL) {
+			pkg_ctx->name = NULL;
 			tree_close_pkg(pkg_ctx);
 			return NULL;
 		}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     a68b9b4ebc840805f504fe681196e7f15737985d
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 11:03:36 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 11:03:36 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=a68b9b4e

libq/atom: fixup USE-dep printing in atom_format

Bug: https://bugs.gentoo.org/724892
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 7e72bf3..acf28be 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -939,12 +939,20 @@ atom_format_r(
 				} else if (!strncmp("USE", fmt, len)) {
 					if (showit || atom->usedeps) {
 						atom_usedep *ud;
-						append_buf(buf, buflen, "%s", "[");
-						for (ud = atom->usedeps; ud != NULL; ud = ud->next)
-							append_buf(buf, buflen, "%s%s%s%s%s%s",
-									MAGENTA, atom_usecond_str[ud->pfx_cond],
-									ud->use, atom_usecond_str[ud->sfx_cond],
-									NORM, ud->next == NULL ? "]" : ",");
+						if (atom->usedeps == NULL) {
+							append_buf(buf, buflen, "%s", "<unset>");
+						} else {
+							if (connected)
+								append_buf(buf, buflen, "%s", "[");
+							for (ud = atom->usedeps; ud != NULL; ud = ud->next)
+								append_buf(buf, buflen, "%s%s%s%s%s%s",
+										MAGENTA, atom_usecond_str[ud->pfx_cond],
+										ud->use, atom_usecond_str[ud->sfx_cond],
+										NORM, ud->next == NULL ? "" :
+										(connected ? "," : " "));
+							if (connected)
+								append_buf(buf, buflen, "%s", "]");
+						}
 					}
 				} else
 					append_buf(buf, buflen, "<BAD:%.*s>", (int)len, fmt);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     f6d31778569b3bb53a743515a8993aa64cdf0bfe
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 10:28:35 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 10:28:35 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=f6d31778

libq/atom: add atom_clone function

atom_clone() does a deep copy of the given atom, such that it lives
completely on its own and needs its own atom_implode() call.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 libq/atom.h |   1 +
 2 files changed, 107 insertions(+)

diff --git a/libq/atom.c b/libq/atom.c
index e79c30a..7e72bf3 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -355,6 +355,112 @@ atom_explode(const char *atom)
 	return ret;
 }
 
+depend_atom *
+atom_clone(depend_atom *atom)
+{
+	depend_atom *ret;
+	char *p;
+	size_t alen;
+	size_t clen = 0;
+	size_t flen = 0;
+	size_t plen = 0;
+	size_t nlen = 0;
+	size_t slen = 0;
+	size_t rlen = 0;
+
+	if (atom->REPO != NULL)
+		rlen = strlen(atom->REPO) + 1;
+	if (atom->SLOT != NULL)
+		slen = strlen(atom->SLOT) + 1;
+	if (atom->CATEGORY != NULL)
+		clen = strlen(atom->CATEGORY) + 1;
+	if (atom->PF != NULL)
+		flen = strlen(atom->PF) + 1;  /* should include PVR */
+	if (atom->P != NULL)
+		plen = strlen(atom->P) + 1;  /* should include PV */
+	if (atom->PN != NULL)
+		nlen = strlen(atom->PN) + 1;
+
+	alen = sizeof(*ret) + clen + flen + plen + nlen + rlen + slen;
+	ret = xmalloc(alen);
+	memset(ret, '\0', sizeof(*ret));
+
+	/* build up main storage pointers, see explode */
+	p = (char *)ret + sizeof(*ret);
+	if (atom->CATEGORY != NULL) {
+		ret->CATEGORY = p;
+		memcpy(ret->CATEGORY, atom->CATEGORY, clen);
+		p += clen;
+	}
+	if (atom->PF != NULL) {
+		ret->PF = p;
+		memcpy(ret->PF, atom->PF, flen);
+		p += flen;
+	}
+	if (atom->PVR > atom->PF && atom->PVR < (atom->PF + flen))
+		ret->PVR = ret->PF + (atom->PVR - atom->PF);
+	if (atom->P != NULL) {
+		ret->P = p;
+		memcpy(ret->P, atom->P, plen);
+		p += plen;
+	}
+	if (atom->PV > atom->P && atom->PV < (atom->P + plen))
+		ret->PV = ret->P + (atom->PV - atom->P);
+	if (atom->PN != NULL) {
+		ret->PN = p;
+		memcpy(ret->PN, atom->PN, nlen);
+		p += nlen;
+	}
+	if (atom->SLOT != NULL) {
+		ret->SLOT = p;
+		memcpy(ret->SLOT, atom->SLOT, slen);
+		p += slen;
+	}
+	if (atom->SUBSLOT > atom->SLOT && atom->SUBSLOT < (atom->SLOT + slen))
+		ret->SUBSLOT = ret->SLOT + (atom->SUBSLOT - atom->SLOT);
+	if (atom->REPO != NULL) {
+		ret->REPO = p;
+		memcpy(ret->REPO, atom->REPO, rlen);
+		p += rlen;
+	}
+
+	ret->blocker = atom->blocker;
+	ret->pfx_op = atom->pfx_op;
+	ret->sfx_op = atom->pfx_op;
+	ret->PR_int = atom->PR_int;
+	ret->letter = atom->letter;
+	ret->slotdep = atom->slotdep;
+
+	if (atom->suffixes != NULL) {
+		for (slen = 0; atom->suffixes[slen].suffix != VER_NORM; slen++)
+			;
+		slen++;
+		ret->suffixes = xmalloc(sizeof(ret->suffixes[0]) * slen);
+		memcpy(ret->suffixes, atom->suffixes, sizeof(ret->suffixes[0]) * slen);
+	}
+
+	if (atom->usedeps) {
+		atom_usedep *w;
+		atom_usedep *n = NULL;
+
+		for (w = atom->usedeps; w != NULL; w = w->next) {
+			nlen = w->use != NULL ? strlen(w->use) + 1 : 0;
+			if (n == NULL) {
+				atom->usedeps = n = xmalloc(sizeof(*n) + nlen);
+			} else {
+				n = n->next = xmalloc(sizeof(*n) + nlen);
+			}
+			n->next = NULL;
+			n->pfx_cond = w->pfx_cond;
+			n->sfx_cond = w->sfx_cond;
+			n->use = (char *)n + sizeof(*n);
+			memcpy(n->use, w->use, nlen);
+		}
+	}
+
+	return ret;
+}
+
 void
 atom_implode(depend_atom *atom)
 {

diff --git a/libq/atom.h b/libq/atom.h
index 1548dd9..aff4548 100644
--- a/libq/atom.h
+++ b/libq/atom.h
@@ -97,6 +97,7 @@ typedef enum {
 } atom_equality;
 
 depend_atom *atom_explode(const char *atom);
+depend_atom *atom_clone(depend_atom *atom);
 void atom_implode(depend_atom *atom);
 atom_equality atom_compare(const depend_atom *a1, const depend_atom *a2);
 atom_equality atom_compare_str(const char * const s1, const char * const s2);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     04978409eb34973d3af6ce0592fd623e82ea6573
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 10:33:59 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 10:33:59 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=04978409

libq/tree: allow tree_foreach_packages to be called multiple times

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index d901fc6..4b9109e 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1293,6 +1293,22 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 	tree_pkg_meta meta;
 	depend_atom *atom = NULL;
 
+	/* re-read the contents, this is necessary to make it possible to
+	 * call this function multiple times
+	 * TODO: generate an internal in-memory tree when cache is enabled */
+	if (ctx->pkgs == NULL || ctx->pkgs[0] == '\0') {
+		int fd = openat(ctx->tree_fd, binpkg_packages, O_RDONLY | O_CLOEXEC);
+		if (!eat_file_fd(fd, &ctx->pkgs, &ctx->pkgslen)) {
+			if (ctx->pkgs != NULL) {
+				free(ctx->pkgs);
+				ctx->pkgs = NULL;
+			}
+			close(fd);
+			return 1;
+		}
+		close(fd);
+	}
+
 	memset(&meta, 0, sizeof(meta));
 
 	do {

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     553eb9afd1a57bbe56bdddd77ddf0bb450ee32b8
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 10:34:54 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 10:34:54 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=553eb9af

libq/tree: rework tree_match_atom to return a list of results

Major change is that multiple results are returned, somewhat controlled
by flags, and that the results are valid until the tree is closed.  Each
match result needs to be closed individually though.

This required some hoops to make it work with Packages file or binpkg
repos.  The latter are largely simulated now, and do not really benefit
from the cheaper point and lookup approach that tree_match_atom tries to
provide.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 181 ++++++++++++++++++++++++++++++++++++++++++++++++++++--------
 libq/tree.h |  21 ++++++-
 2 files changed, 177 insertions(+), 25 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 4b9109e..d313f3b 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -388,14 +388,11 @@ tree_filter_pkg(const struct dirent *de)
 tree_pkg_ctx *
 tree_open_pkg(tree_cat_ctx *cat_ctx, const char *name)
 {
-	tree_pkg_ctx *pkg_ctx = xmalloc(sizeof(*pkg_ctx));
+	tree_pkg_ctx *pkg_ctx = xzalloc(sizeof(*pkg_ctx));
 	pkg_ctx->name = name;
-	pkg_ctx->slot = NULL;
 	pkg_ctx->repo = cat_ctx->ctx->repo;
 	pkg_ctx->fd = -1;
 	pkg_ctx->cat_ctx = cat_ctx;
-	pkg_ctx->atom = NULL;
-	pkg_ctx->meta = NULL;
 
 	/* see if this pkg matches the query, here we can finally check
 	 * version conditions like >=, etc. */
@@ -1405,12 +1402,14 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 		match_key(IUSE);
 		match_key(KEYWORDS);
 		match_key(LICENSE);
-		match_key2(MD5, _md5_);
-		match_key2(SHA1, _eclasses_);
+		match_key(MD5);
+		match_key(SHA1);
 		match_key(RDEPEND);
 		match_key(SLOT);
 		match_key(USE);
 		match_key(PDEPEND);
+		match_key2(REPO, repository);
+		match_key(SIZE);
 #undef match_key
 #undef match_key2
 		}
@@ -1430,6 +1429,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 	/* ensure we don't free a garbage pointer */
 	ctx->repo = NULL;
 	ctx->do_sort = false;
+	ctx->pkgs[0] = '\0';
 
 	return ret;
 }
@@ -1580,34 +1580,167 @@ tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms)
 	return state.cpf;
 }
 
-tree_pkg_ctx *
-tree_match_atom(tree_ctx *ctx, depend_atom *a)
+struct tree_match_pkgs_cb_ctx {
+	int flags;
+	tree_match_ctx *ret;
+};
+
+static int
+tree_match_atom_packages_cb(tree_pkg_ctx *ctx, void *priv)
+{
+	struct tree_match_pkgs_cb_ctx *rctx = priv;
+	depend_atom *a;
+	tree_match_ctx *n;
+
+	/* skip anything after finding first match */
+	if (rctx->flags & TREE_MATCH_FIRST && rctx->ret != NULL)
+		return 1;
+
+	a = tree_get_atom(ctx, rctx->flags & TREE_MATCH_FULL_ATOM);
+
+	/* skip virtual category if not requested */
+	if (!(rctx->flags & TREE_MATCH_VIRTUAL) &&
+			strcmp(a->CATEGORY, "virtual") == 0)
+		return 1;
+
+	n = xzalloc(sizeof(tree_match_ctx));
+	n->free_atom = true;
+	n->atom = atom_clone(a);
+	if (rctx->flags & TREE_MATCH_METADATA) {
+		n->meta = xmalloc(sizeof(*n->meta));
+		/* for Packages, all pointers to meta here are to the in memory
+		 * copy of the Packages file, so these pointers can just be
+		 * copied since the tree has to remain open, thus the pointers
+		 * will stay valid */
+		memcpy(n->meta, ctx->cat_ctx->ctx->pkgs, sizeof(*n->meta));
+	}
+
+	n->next = rctx->ret;
+	rctx->ret = n;
+
+	return 0;
+}
+
+static int
+tree_match_atom_binpkg_cb(tree_pkg_ctx *ctx, void *priv)
+{
+	struct tree_match_pkgs_cb_ctx *rctx = priv;
+	depend_atom *a;
+	tree_match_ctx *n;
+
+	/* skip anything after finding first match */
+	if (rctx->flags & TREE_MATCH_FIRST && rctx->ret != NULL)
+		return 1;
+
+	a = tree_get_atom(ctx, rctx->flags & TREE_MATCH_FULL_ATOM);
+
+	/* skip virtual category if not requested */
+	if (!(rctx->flags & TREE_MATCH_VIRTUAL) &&
+			strcmp(a->CATEGORY, "virtual") == 0)
+		return 1;
+
+	n = xzalloc(sizeof(tree_match_ctx));
+	n->free_atom = true;
+	n->atom = atom_clone(a);
+	if (rctx->flags & TREE_MATCH_METADATA)
+		n->meta = tree_pkg_read(ctx);
+
+	n->next = rctx->ret;
+	rctx->ret = n;
+
+	return 0;
+}
+
+tree_match_ctx *
+tree_match_atom(tree_ctx *ctx, depend_atom *query, int flags)
 {
 	tree_cat_ctx *cat_ctx;
 	tree_pkg_ctx *pkg_ctx;
+	tree_match_ctx *ret = NULL;
 	depend_atom *atom;
 
+	if (ctx->cachetype == CACHE_PACKAGES) {
+		struct tree_match_pkgs_cb_ctx rctx;
+		/* Packages needs to be serviced separately because it doesn't
+		 * use a tree internally, but reads off of the Packages file */
+		rctx.flags = flags;
+		rctx.ret = NULL;
+		ctx->query_atom = query;
+		tree_foreach_packages(ctx, tree_match_atom_packages_cb, &rctx);
+		ctx->query_atom = NULL;
+		return rctx.ret;
+	} else if (ctx->cachetype == CACHE_BINPKGS) {
+		struct tree_match_pkgs_cb_ctx rctx;
+		/* this sulks, but binpkgs modify the pkg_ctx->name to strip off
+		 * .tbz2, and that makes it non-reusable */
+		rctx.flags = flags;
+		rctx.ret = NULL;
+		tree_foreach_pkg(ctx, tree_match_atom_binpkg_cb, &rctx, true, query);
+		return rctx.ret;
+	}
+
+	/* activate cache for future lookups, tree_match_atom relies on
+	 * cache behaviour from tree, which means all categories and
+	 * packages remain in memory until tree_close is being called */
 	if (ctx->cache.categories == NULL)
 		ctx->cache.categories = create_set();
 
-	if (a->P == NULL) {
-		return NULL;
-	} else if (a->CATEGORY == NULL) {
-		/* loop through all cats and recurse */
-		/* TODO: some day */
-		return NULL;
+	ctx->do_sort = true;     /* sort uses buffer, which cache relies on */
+	ctx->query_atom = NULL;  /* ensure the cache contains ALL pkgs */
+
+#define search_cat(C) \
+{ \
+	while ((pkg_ctx = tree_next_pkg(C)) != NULL) { \
+		atom = tree_get_atom(pkg_ctx, \
+				query->SLOT != NULL || flags & TREE_MATCH_FULL_ATOM); \
+fprintf(stderr, "fbg: %s\n", atom_to_string(atom)); \
+		if (flags & TREE_MATCH_VIRTUAL || \
+				strcmp(atom->CATEGORY, "virtual") != 0) \
+			if (atom_compare(atom, query) == EQUAL) { \
+				tree_match_ctx *n; \
+				n = xzalloc(sizeof(tree_match_ctx)); \
+				n->free_atom = false; \
+				n->atom = atom; \
+				if (flags & TREE_MATCH_METADATA) \
+					n->meta = tree_pkg_read(pkg_ctx); \
+				n->next = ret; \
+				ret = n; \
+			} \
+			if (flags & TREE_MATCH_FIRST && ret != NULL) \
+				break; \
+	} \
+	C->pkg_cur = 0;  /* reset to allow another traversal */ \
+}
+
+	if (query->CATEGORY == NULL) {
+		/* loop through all cats */
+		while ((cat_ctx = tree_next_cat(ctx)) != NULL) {
+			search_cat(cat_ctx);
+			if (ret != NULL && flags & TREE_MATCH_FIRST)
+				break;
+		}
+		/* allow running again through the cats */
+		ctx->cat_cur = 0;
 	} else {
 		/* try CAT, and PN for latest version */
-		if ((cat_ctx = tree_open_cat(ctx, a->CATEGORY)) == NULL)
-			return NULL;
-		ctx->do_sort = true;     /* sort uses buffer, which cache relies on */
-		ctx->query_atom = NULL;  /* ensure the cache contains ALL pkgs */
-		while ((pkg_ctx = tree_next_pkg(cat_ctx)) != NULL) {
-			atom = tree_get_atom(pkg_ctx, a->SLOT != NULL);
-			if (atom_compare(atom, a) == EQUAL)
-				return pkg_ctx;
-		}
+		if ((cat_ctx = tree_open_cat(ctx, query->CATEGORY)) != NULL)
+			search_cat(cat_ctx);
+	}
 
-		return NULL;
+	return ret;
+}
+
+void
+tree_match_close(tree_match_ctx *match)
+{
+	tree_match_ctx *w;
+
+	for (w = NULL; match != NULL; match = w) {
+		w = match->next;
+		if (match->free_atom)
+			atom_implode(match->atom);
+		if (match->meta != NULL)
+			free(match->meta);
+		free(match);
 	}
 }

diff --git a/libq/tree.h b/libq/tree.h
index eaee7ad..900b998 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -18,6 +18,7 @@ typedef struct tree_cat_ctx      tree_cat_ctx;
 typedef struct tree_pkg_ctx      tree_pkg_ctx;
 typedef struct tree_pkg_meta     tree_pkg_meta;
 typedef struct tree_metadata_xml tree_metadata_xml;
+typedef struct tree_match_ctx    tree_match_ctx;
 
 /* tree context */
 struct tree_ctx {
@@ -100,6 +101,9 @@ struct tree_pkg_meta {
 	char *Q_USE;
 	char *Q_EPREFIX;
 	char *Q_repository;
+	char *Q_MD5;
+	char *Q_SHA1;
+#define Q_SIZE Q_SRC_URI
 	/* These are MD5-Cache only */
 	char *Q__eclasses_;
 	char *Q__md5_;
@@ -113,6 +117,15 @@ struct tree_metadata_xml {
 	} *email;
 };
 
+/* used with tree_match_atom, both atom and meta are fully materialised
+ * (populated and deep copied) when set */
+struct tree_match_ctx {
+	depend_atom *atom;
+	tree_pkg_meta *meta;
+	tree_match_ctx *next;
+	int free_atom;
+};
+
 /* foreach pkg callback function signature */
 typedef int (tree_pkg_cb)(tree_pkg_ctx *, void *priv);
 
@@ -138,6 +151,12 @@ int tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 	tree_foreach_pkg(ctx, cb, priv, true, query);
 set *tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms);
 depend_atom *tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete);
-tree_pkg_ctx *tree_match_atom(tree_ctx *t, depend_atom *a);
+tree_match_ctx *tree_match_atom(tree_ctx *t, depend_atom *q, int flags);
+#define TREE_MATCH_FULL_ATOM   1<<1
+#define TREE_MATCH_METADATA    1<<2
+#define TREE_MATCH_FIRST       1<<3
+#define TREE_MATCH_VIRTUAL     1<<4
+#define TREE_MATCH_DEFAULT     TREE_MATCH_VIRTUAL
+void tree_match_close(tree_match_ctx *t);
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     3794f21d3da7f182c85b63e87f0e073b5df3de18
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 10:32:27 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 10:32:27 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=3794f21d

libq/tree: have tree_read_file_binpkg populate some meta fields

The SHA1 and SIZE fields might be necessary, so psuedo fill them in
here, as we don't have a meta that contains them, except the file
itself.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index 7fbb739..d901fc6 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -17,6 +17,7 @@
 
 #include "atom.h"
 #include "eat_file.h"
+#include "hash.h"
 #include "rmspace.h"
 #include "scandirat.h"
 #include "set.h"
@@ -982,10 +983,48 @@ static tree_pkg_meta *
 tree_read_file_binpkg(tree_pkg_ctx *pkg_ctx)
 {
 	tree_pkg_meta *m = xzalloc(sizeof(tree_pkg_meta));
+	int newfd = dup(pkg_ctx->fd);
 
 	xpak_process_fd(pkg_ctx->fd, true, m, tree_read_file_binpkg_xpak_cb);
 	pkg_ctx->fd = -1;  /* closed by xpak_process_fd */
 
+	/* fill in some properties which are not available, but would be in
+	 * Packages, and used to verify the package ... this is somewhat
+	 * fake, but allows to transparantly use a dir of binpkgs */
+	if (newfd != -1) {
+		size_t fsize;
+		size_t needlen = 40 + 1 + 19 + 1;
+		size_t pos = (size_t)m->Q__eclasses_;
+		size_t len = (size_t)m->Q__md5_;
+
+		if (len - pos < needlen) {
+			char *old_data = m->Q__data;
+			len += ((needlen / BUFSIZ) + 1) * BUFSIZ;
+			m->Q__data = xrealloc(m->Q__data, len);
+			m->Q__md5_ = (char *)len;
+
+			/* re-position existing keys */
+			if (old_data != NULL && m->Q__data != old_data) {
+				char **newdata = (char **)m;
+				int elems = sizeof(tree_pkg_meta) / sizeof(char *);
+				while (elems-- > 1)  /* skip Q__data itself */
+					if (newdata[elems] != NULL)
+						newdata[elems] =
+							m->Q__data + (newdata[elems] - old_data);
+			}
+		}
+
+		m->Q_SHA1 = m->Q__data + pos;
+		m->Q_SIZE = m->Q_SHA1 + 40 + 1;
+		pos += needlen;
+		m->Q__eclasses_ = (char *)pos;
+
+		lseek(newfd, 0, SEEK_SET);  /* reposition at the whole file */
+		if (hash_multiple_file_fd(newfd, NULL, m->Q_SHA1, NULL, NULL,
+				NULL, NULL, &fsize, HASH_SHA1) == 0)
+			snprintf(m->Q_SIZE, 19 + 1, "%zu", fsize);
+	}
+
 	return m;
 }
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     1cffb01452f2553747be26778ef4f84425e45554
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May 25 10:30:52 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May 25 10:30:52 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=1cffb014

libq/tree: slightly optimise tree_open_pkg

don't construct a new path, but open at the existing filedescriptor
instead.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index ebcc133..7fbb739 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -27,7 +27,7 @@
 #include <xalloc.h>
 
 static int tree_pkg_compar(const void *l, const void *r);
-static tree_pkg_ctx * tree_next_pkg_int(tree_cat_ctx *cat_ctx);
+static tree_pkg_ctx *tree_next_pkg_int(tree_cat_ctx *cat_ctx);
 static void tree_close_meta(tree_pkg_meta *cache);
 
 static tree_ctx *
@@ -132,18 +132,19 @@ tree_ctx *
 tree_open_binpkg(const char *sroot, const char *spkg)
 {
 	tree_ctx *ret = tree_open_int(sroot, spkg, true);
-	char buf[_Q_PATH_MAX];
+	int fd;
 
 	if (ret != NULL) {
 		ret->cachetype = CACHE_BINPKGS;
 
-		snprintf(buf, sizeof(buf), "%s%s/%s", sroot, spkg, binpkg_packages);
-		if (eat_file(buf, &ret->pkgs, &ret->pkgslen)) {
+		fd = openat(ret->tree_fd, binpkg_packages, O_RDONLY | O_CLOEXEC);
+		if (eat_file_fd(fd, &ret->pkgs, &ret->pkgslen)) {
 			ret->cachetype = CACHE_PACKAGES;
 		} else if (ret->pkgs != NULL) {
 			free(ret->pkgs);
 			ret->pkgs = NULL;
 		}
+		close(fd);
 	}
 
 	return ret;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     2dce04019d38a49f7930db13e62dd47ecf5f9471
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun May 17 12:32:52 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun May 17 12:32:52 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=2dce0401

hash_multiple_file_fd: return success status

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/hash.c | 29 ++++++++++++++++++-----------
 libq/hash.h |  4 ++--
 2 files changed, 20 insertions(+), 13 deletions(-)

diff --git a/libq/hash.c b/libq/hash.c
index a174a0c..a51ea6a 100644
--- a/libq/hash.c
+++ b/libq/hash.c
@@ -113,7 +113,7 @@ hash_hex(char *out, const unsigned char *buf, const int length)
  * left untouched, e.g. they can be NULL.  The number of bytes read from
  * the file pointed to by fname is returned in the flen argument.
  */
-void
+int
 hash_multiple_file_fd(
 		int fd,
 		char *md5,
@@ -148,7 +148,7 @@ hash_multiple_file_fd(
 #endif
 
 	if ((f = fdopen(fd, "r")) == NULL)
-		return;
+		return -1;
 
 #ifdef HAVE_SSL
 	MD5_Init(&m5);
@@ -164,6 +164,7 @@ hash_multiple_file_fd(
 	blake2b_init(&bl2b, BLAKE2B_OUTBYTES);
 #endif
 
+	*flen = 0;
 	while ((len = fread(data, 1, sizeof(data), f)) > 0) {
 		*flen += len;
 #pragma omp parallel sections
@@ -291,9 +292,10 @@ hash_multiple_file_fd(
 	}
 
 	fclose(f);
+	return 0;
 }
 
-void
+int
 hash_multiple_file_at_cb(
 		int pfd,
 		const char *fname,
@@ -307,19 +309,23 @@ hash_multiple_file_at_cb(
 		size_t *flen,
 		int hashes)
 {
+	int ret;
 	int fd = openat(pfd, fname, O_RDONLY | O_CLOEXEC);
-	if (fd == -1) {
-		*flen = 0;
-		return;
-	}
 
-	if (cb != NULL)
+	if (fd == -1)
+		return -1;
+
+	if (cb != NULL) {
 		fd = cb(fd, fname);
+		if (fd == -1)
+			return -1;
+	}
 
-	hash_multiple_file_fd(fd, md5, sha1, sha256, sha512,
+	ret = hash_multiple_file_fd(fd, md5, sha1, sha256, sha512,
 			whrlpl, blak2b, flen, hashes);
 
 	close(fd);
+	return ret;
 }
 
 static char _hash_file_buf[128 + 1];
@@ -335,10 +341,11 @@ hash_file_at_cb(int pfd, const char *fname, int hash, hash_cb_t cb)
 		case HASH_SHA512:
 		case HASH_WHIRLPOOL:
 		case HASH_BLAKE2B:
-			hash_multiple_file_at_cb(pfd, fname, cb,
+			if (hash_multiple_file_at_cb(pfd, fname, cb,
 					_hash_file_buf, _hash_file_buf, _hash_file_buf,
 					_hash_file_buf, _hash_file_buf, _hash_file_buf,
-					&dummy, hash);
+					&dummy, hash) != 0)
+				return NULL;
 			break;
 		default:
 			return NULL;

diff --git a/libq/hash.h b/libq/hash.h
index 204da5f..f85080d 100644
--- a/libq/hash.h
+++ b/libq/hash.h
@@ -30,10 +30,10 @@ enum hash_impls {
 typedef int (*hash_cb_t) (int, const char *);
 
 void hash_hex(char *out, const unsigned char *buf, const int length);
-void hash_multiple_file_fd(
+int hash_multiple_file_fd(
 		int fd, char *md5, char *sha1, char *sha256, char *sha512,
 		char *whrlpl, char *blak2b, size_t *flen, int hashes);
-void hash_multiple_file_at_cb(
+int hash_multiple_file_at_cb(
 		int pfd, const char *fname, hash_cb_t cb, char *md5,
 		char *sha1, char *sha256, char *sha512, char *whrlpl,
 		char *blak2b, size_t *flen, int hashes);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     0d354ac463a11013068f61cb7c1db987048f2a12
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun May 17 10:03:34 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun May 17 10:03:34 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=0d354ac4

xpak: change xpak_process to return start position of XPAKPACK

Allow to get the size of the archive by a call to xpak_process, or
failure if size is -1.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xpak.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/libq/xpak.c b/libq/xpak.c
index 59c541d..0e8bd7e 100644
--- a/libq/xpak.c
+++ b/libq/xpak.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2020 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
@@ -46,9 +46,11 @@
 typedef struct {
 	void *ctx;
 	FILE *fp;
+	unsigned int xpakstart;  /* offset in the file for XPAKPACK */
 	unsigned int index_len;
 	unsigned int data_len;
-	char *index, *data;
+	char *index;
+	char *data;
 } _xpak_archive;
 
 static void _xpak_walk_index(
@@ -119,6 +121,7 @@ static _xpak_archive *_xpak_open(const int fd)
 
 				if (fseek(ret.fp, -(xpaklen + TBZ2_FOOTER_LEN), SEEK_END) == 0)
 				{
+					ret.xpakstart = (unsigned int)ftell(ret.fp);
 					if (fread(buf, 1, XPAK_START_LEN, ret.fp) != XPAK_START_LEN)
 						goto close_and_ret;
 					if (memcmp(buf, XPAK_START_MSG, XPAK_START_MSG_LEN) == 0)
@@ -129,6 +132,7 @@ static _xpak_archive *_xpak_open(const int fd)
 		warn("Not an xpak file");
 		goto close_and_ret;
 	}
+	ret.xpakstart = 0;  /* pure xpak file */
 
 setup_lens:
 	/* calc index and data sizes */
@@ -165,7 +169,7 @@ xpak_process_fd(
 
 	x = _xpak_open(fd);
 	if (!x)
-		return 1;
+		return -1;
 
 	x->ctx = ctx;
 	x->index = buf;
@@ -196,7 +200,7 @@ xpak_process_fd(
 	if (get_data)
 		free(x->data);
 
-	return 0;
+	return x->xpakstart;
 }
 
 int
@@ -215,7 +219,7 @@ xpak_process(
 		return -1;
 
 	ret = xpak_process_fd(fd, get_data, ctx, func);
-	if (ret != 0)
+	if (ret < 0)
 		warn("Unable to open file '%s'", file);
 
 	return ret;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     38bb361b2a8ed4bcf9b6f9ee3eb48ccd815bb382
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Feb  3 13:15:48 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Feb  3 13:15:48 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=38bb361b

libq/tree: be more conservative with mem-clearing

Previous fix cleared unnecessary amounts of memory (immediately
overwritten by fread()), use a memset on the area of memory required.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 8424459..1c2a54e 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -689,8 +689,9 @@ tree_read_file_md5(tree_pkg_ctx *pkg_ctx)
 		goto err;
 
 	len = sizeof(*ret) + s.st_size + 1;
-	ret = xzalloc(len);
-	ptr = (char*)ret + sizeof(*ret);
+	ret = xmalloc(len);
+	memset(ret, 0, sizeof(*ret));
+	ptr = (char *)ret + sizeof(*ret);
 	if ((off_t)fread(ptr, 1, s.st_size, f) != s.st_size)
 		goto err;
 	ptr[s.st_size] = '\0';
@@ -802,7 +803,8 @@ tree_read_file_ebuild(tree_pkg_ctx *pkg_ctx)
 		goto err;
 
 	len = sizeof(*ret) + s.st_size + 1;
-	ret = xzalloc(len);
+	ret = xmalloc(len);
+	memset(ret, 0, sizeof(*ret));
 	p = (char *)ret + sizeof(*ret);
 	if ((off_t)fread(p, 1, s.st_size, f) != s.st_size)
 		goto err;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     a7182e09e90bcd37cca9f56639f604a809ab4c1d
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Feb  3 13:07:47 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Feb  3 13:07:47 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=a7182e09

libq/tree: ensure meta fields are clear when not set

For some reason we went with uncleared pointers.  Probably because we
only requested things that should/would exist.  However, for sanity, and
probably bug #701470, return NULL pointers for anything not set (in
metadata).

Bug: https://bugs.gentoo.org/show_bug.cgi?id=701470
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 9df3541..8424459 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -621,7 +621,7 @@ tree_read_file_pms(tree_pkg_ctx *pkg_ctx)
 		goto err;
 
 	len = sizeof(*ret) + s.st_size + 1;
-	ret = xmalloc(len);
+	ret = xzalloc(len);
 	ptr = (char*)ret + sizeof(*ret);
 	if ((off_t)fread(ptr, 1, s.st_size, f) != s.st_size)
 		goto err;
@@ -689,7 +689,7 @@ tree_read_file_md5(tree_pkg_ctx *pkg_ctx)
 		goto err;
 
 	len = sizeof(*ret) + s.st_size + 1;
-	ret = xmalloc(len);
+	ret = xzalloc(len);
 	ptr = (char*)ret + sizeof(*ret);
 	if ((off_t)fread(ptr, 1, s.st_size, f) != s.st_size)
 		goto err;
@@ -802,7 +802,7 @@ tree_read_file_ebuild(tree_pkg_ctx *pkg_ctx)
 		goto err;
 
 	len = sizeof(*ret) + s.st_size + 1;
-	ret = xmalloc(len);
+	ret = xzalloc(len);
 	p = (char *)ret + sizeof(*ret);
 	if ((off_t)fread(p, 1, s.st_size, f) != s.st_size)
 		goto err;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     982ea6b9dcea2a86d3772c99cff9ada0c400bf29
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 26 19:30:45 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 26 19:30:45 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=982ea6b9

libq/xpak: fix Coverity 125939 Time of check time of use

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xpak.c | 38 +++++++++++++++++++++++---------------
 1 file changed, 23 insertions(+), 15 deletions(-)

diff --git a/libq/xpak.c b/libq/xpak.c
index 90a3570..59c541d 100644
--- a/libq/xpak.c
+++ b/libq/xpak.c
@@ -223,7 +223,7 @@ xpak_process(
 
 static void
 _xpak_add_file(
-		int dir_fd,
+		int fd,
 		const char *filename,
 		struct stat *st,
 		FILE *findex,
@@ -236,7 +236,7 @@ _xpak_add_file(
 	unsigned char intbuf[4];
 	unsigned char *p = intbuf;
 	const char *basefile;
-	int fd, in_len;
+	int in_len;
 
 	basefile = basename(filename);
 
@@ -259,28 +259,24 @@ _xpak_add_file(
 
 	/* now open the file, get (data_len),
 	 * and append the file to the data file */
-	fd = openat(dir_fd, filename, O_RDONLY|O_CLOEXEC);
-	if (fd < 0) {
- open_fail:
+	if ((fin = fdopen(fd, "r")) == NULL) {
 		warnp("could not open for reading: %s", filename);
- fake_data_len:
 		WRITE_BE_INT32(p, 0);
 		fwrite(p, 1, 4, findex);
 		return;
 	}
-	fin = fdopen(fd, "r");
-	if (!fin) {
-		close(fd);
-		goto open_fail;
-	}
+
 	in_len = st->st_size;
 	/* the xpak format can only store files whose size is a 32bit int
 	 * so we have to make sure we don't store a big file */
 	if (in_len != st->st_size) {
 		warnf("File is too big: %zu", (size_t)st->st_size);
 		fclose(fin);
-		goto fake_data_len;
+		WRITE_BE_INT32(p, 0);
+		fwrite(p, 1, 4, findex);
+		return;
 	}
+
 	WRITE_BE_INT32(p, in_len);
 	fwrite(p, 1, 4, findex);
 	copy_file(fin, fdata);
@@ -333,6 +329,8 @@ xpak_create(
 
 	index_len = data_len = 0;
 	for (i = 0; i < argc; ++i) {
+		int fd;
+
 		if (fstatat(dir_fd, argv[i], &st, 0)) {
 			warnp("fstatat(%s) failed", argv[i]);
 			continue;
@@ -344,22 +342,32 @@ xpak_create(
 			for (fidx = 0; fidx < numfiles; ++fidx) {
 				int ret = snprintf(path, sizeof(path), "%s/%s",
 						argv[i], dir[fidx]->d_name);
+
 				if (ret < 0 || (size_t)ret >= sizeof(path)) {
 					warn("skipping path too long: %s/%s",
 							argv[i], dir[fidx]->d_name);
 					continue;
 				}
-				if (stat(path, &st) < 0) {
+
+				fd = openat(dir_fd, path, O_RDONLY|O_CLOEXEC);
+				if (fd < 0 || fstat(fd, &st) < 0) {
 					warnp("could not read %s", path);
 					continue;
 				}
-				_xpak_add_file(dir_fd, path, &st,
+				_xpak_add_file(fd, path, &st,
 						findex, &index_len, fdata, &data_len, verbose);
+				close(fd);
 			}
 			scandir_free(dir, numfiles);
 		} else if (S_ISREG(st.st_mode)) {
-			_xpak_add_file(dir_fd, argv[i], &st,
+			fd = openat(dir_fd, argv[i], O_RDONLY|O_CLOEXEC);
+			if (fd < 0 || fstat(fd, &st) < 0) {
+				warnp("could not read %s", path);
+				continue;
+			}
+			_xpak_add_file(fd, argv[i], &st,
 					findex, &index_len, fdata, &data_len, verbose);
+			close(fd);
 		} else
 			warn("Skipping non file/directory '%s'", argv[i]);
 	}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     0501fd2dfe83fc29d44a5b45586fef33e63800f5
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Jan 22 19:42:24 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Jan 22 19:42:24 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=0501fd2d

libq/dep: fix Coverity 125901 Explicit null dereferenced

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/dep.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/dep.c b/libq/dep.c
index 0507326..3667ae5 100644
--- a/libq/dep.c
+++ b/libq/dep.c
@@ -178,7 +178,7 @@ dep_grow_tree(const char *depend)
 				break;
 			_maybe_consume_word(DEP_NORM);
 
-			if (curr_node->parent == NULL) {
+			if (curr_node == NULL || curr_node->parent == NULL) {
 				warnf("Group lacks a parent");
 				goto error_out;
 			}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     f7f550ce23ea6c0db4ce33a816aa61e0e815cf91
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Jan 22 19:29:30 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Jan 22 19:29:30 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=f7f550ce

libq/eat_file: fix Coverity 125891 Ignoring number of bytes read

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/eat_file.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libq/eat_file.c b/libq/eat_file.c
index 6deabe2..85ba691 100644
--- a/libq/eat_file.c
+++ b/libq/eat_file.c
@@ -40,7 +40,7 @@ eat_file_fd(int fd, char **bufptr, size_t *bufsize)
 		/* We assume a min allocation size so that repeat calls don't
 		 * hit ugly ramp ups -- if you read a file that is 1 byte, then
 		 * 5 bytes, then 10 bytes, then 20 bytes, ... you'll allocate
-		 * constantly.  So we round up a few pages as wasiting virtual
+		 * constantly.  So we round up a few pages as wasting virtual
 		 * memory is cheap when it is unused.  */
 		*bufsize = ((read_size + 1) + BUFSIZE - 1) & -BUFSIZE;
 		*bufptr = xrealloc(*bufptr, *bufsize);
@@ -55,9 +55,9 @@ eat_file_fd(int fd, char **bufptr, size_t *bufsize)
 				return false;
 			buf[read_size] = '\0';
 		} else {
-			if (read(fd, buf, read_size) == 0)
+			if ((read_size = read(fd, buf, read_size)) <= 0)
 				return false;
-			buf[read_size - 1] = '\0';
+			buf[read_size] = '\0';
 		}
 	}
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     981f0a47acc281bcfec4faa966d0bfb447b28ddd
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Jan 20 19:53:47 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Jan 20 19:53:47 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=981f0a47

libq/colors: fix Coverity 183476 Buffer not null terminated

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/colors.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/libq/colors.c b/libq/colors.c
index 88664c3..9a48e2e 100644
--- a/libq/colors.c
+++ b/libq/colors.c
@@ -102,8 +102,9 @@ color_remap(void)
 					int found = 0;
 					for (n = 0; n < ARRAY_SIZE(color_pairs); n++) {
 						if (strcmp(color_pairs[n].name, p) == 0) {
-							strncpy(color_pairs[i].value,
-									color_pairs[n].origval, CPAIR_VALUE_LEN);
+							snprintf(color_pairs[i].value,
+									sizeof(color_pairs[i].value),
+									"%s", color_pairs[n].origval);
 							found = 1;
 							break;
 						}
@@ -111,7 +112,7 @@ color_remap(void)
 
 					if (!found)
 						snprintf(color_pairs[i].value,
-								sizeof(color_pairs[i].origval), "\e[%s", p);
+								sizeof(color_pairs[i].value), "\e[%s", p);
 				}
 			}
 		}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     3cb097b70c46b499c7736d50ea6b683a8e4ac2b2
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Jan 20 19:33:35 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Jan 20 19:33:35 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=3cb097b7

libq/tree: fix Coverity 206560 Resource leak

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index 8da8120..9df3541 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1366,6 +1366,9 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 		tree_close_cat(cat);
 	}
 
+	if (atom != NULL)
+		atom_implode(atom);
+
 	/* ensure we don't free a garbage pointer */
 	ctx->repo = NULL;
 	ctx->do_sort = false;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     5cdb5572185636cd5ab056241635d33b1b85fc98
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 19:18:48 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 19:18:48 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=5cdb5572

libq/tree: help Coverity 206538 String not null terminated

Coverity doesn't see that zalloc would have null-terminated.  Upside,
replace zalloc with malloc and just clear one (relevant) byte.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index c8ada5c..8da8120 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -689,10 +689,11 @@ tree_read_file_md5(tree_pkg_ctx *pkg_ctx)
 		goto err;
 
 	len = sizeof(*ret) + s.st_size + 1;
-	ret = xzalloc(len);
+	ret = xmalloc(len);
 	ptr = (char*)ret + sizeof(*ret);
 	if ((off_t)fread(ptr, 1, s.st_size, f) != s.st_size)
 		goto err;
+	ptr[s.st_size] = '\0';
 
 	/* We have a block of key=value\n data.
 	 * KEY=VALUE\n

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     f7d28b1055bc515d9e510329c2840c1993f75654
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 19:08:44 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 19:08:44 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=f7d28b10

libq/tree: fix Coverity 206542 Resource leak

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index dc6ee61..c8ada5c 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -553,7 +553,10 @@ tree_next_pkg(tree_cat_ctx *cat_ctx)
 		} while (ret == NULL);
 	} else if (ctx->cachetype == CACHE_BINPKGS) {
 		char *p = NULL;
+		ret = NULL;
 		do {
+			if (ret != NULL)
+				tree_close_pkg(ret);
 			ret = tree_next_pkg_int(cat_ctx);
 		} while (ret != NULL && (p = strstr(ret->name, ".tbz2")) == NULL);
 		if (p != NULL)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     c4e0ef9df105c2a1459d283cd3d977fcd31d363d
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 18:14:07 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 18:14:07 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=c4e0ef9d

libq/tree: fix Coverity 206551 String not null terminated

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index 2efdb7c..9645237 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -618,10 +618,11 @@ tree_read_file_pms(tree_pkg_ctx *pkg_ctx)
 		goto err;
 
 	len = sizeof(*ret) + s.st_size + 1;
-	ret = xzalloc(len);
+	ret = xmalloc(len);
 	ptr = (char*)ret + sizeof(*ret);
 	if ((off_t)fread(ptr, 1, s.st_size, f) != s.st_size)
 		goto err;
+	ptr[s.st_size] = '\0';
 
 	ret->Q_DEPEND = ptr;
 #define next_line(curr, next) \

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     00e2a72d50acf329a128b81f116e037c82f27c72
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 18:30:49 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 18:30:49 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=00e2a72d

libq/set: fix Coverity 206548 Sizeof not portable

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/set.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/set.c b/libq/set.c
index ceb6f47..88b5876 100644
--- a/libq/set.c
+++ b/libq/set.c
@@ -252,7 +252,7 @@ list_set(set *q, char ***l)
 	set_elem *w;
 	char **ret;
 
-	ret = *l = xmalloc(sizeof(char **) * (q->len + 1));
+	ret = *l = xmalloc(sizeof(char *) * (q->len + 1));
 	for (i = 0; i < _SET_HASH_SIZE; i++) {
 		for (w = q->buckets[i]; w != NULL; w = w->next) {
 			*ret = w->name;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     1c6dcbf136e0cfd7613dc696d482028453ee306b
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 18:15:56 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 18:15:56 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=1c6dcbf1

libq/tree: help for Coverity 206550 String not null terminated

the zalloc on buffer + 1 size should ensure null-termination, but
Coverity doesn't quite see this, so explicitly null-terminate the
string, thereby slightly optimising zalloc away for plain malloc

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index 9645237..dc6ee61 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1164,13 +1164,14 @@ tree_pkg_metadata(tree_pkg_ctx *pkg_ctx)
 	}
 
 	len = sizeof(*ret) + s.st_size + 1;
-	p = xbuf = xzalloc(len);
+	p = xbuf = xmalloc(len);
 	if ((off_t)fread(p, 1, s.st_size, f) != s.st_size) {
 		free(p);
 		fclose(f);
 		pkg_ctx->fd = -1;
 		return NULL;
 	}
+	p[s.st_size] = '\0';
 
 	ret = xmalloc(sizeof(*ret));
 	ret->email = NULL;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     c8bf4b933d88520d7a9ad6f856412e62772f044f
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 16:31:23 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 16:31:23 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=c8bf4b93

libq/tree: fix Coverity 206560 Resource leak

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 52 +++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 37 insertions(+), 15 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 94f9665..2efdb7c 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -138,8 +138,12 @@ tree_open_binpkg(const char *sroot, const char *spkg)
 		ret->cachetype = CACHE_BINPKGS;
 
 		snprintf(buf, sizeof(buf), "%s%s/%s", sroot, spkg, binpkg_packages);
-		if (eat_file(buf, &ret->pkgs, &ret->pkgslen))
+		if (eat_file(buf, &ret->pkgs, &ret->pkgslen)) {
 			ret->cachetype = CACHE_PACKAGES;
+		} else if (ret->pkgs != NULL) {
+			free(ret->pkgs);
+			ret->pkgs = NULL;
+		}
 	}
 
 	return ret;
@@ -1237,10 +1241,12 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 
 	/* reused for every entry */
 	tree_cat_ctx *cat = NULL;
-	tree_pkg_ctx *pkg = xzalloc(sizeof(tree_pkg_ctx));
-	tree_pkg_meta *meta = xzalloc(sizeof(tree_pkg_meta));
+	tree_pkg_ctx pkg;
+	tree_pkg_meta meta;
 	depend_atom *atom = NULL;
 
+	memset(&meta, 0, sizeof(meta));
+
 	do {
 		/* find next line */
 		c = NULL;
@@ -1257,32 +1263,41 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 			if (atom != NULL) {
 				size_t pkgnamelen;
 
+				memset(&pkg, 0, sizeof(pkg));
+
 				/* store meta ptr in repo->pkgs, such that get_pkg_meta
 				 * can grab it from there (for free) */
-				ctx->pkgs = (char *)meta;
+				c = ctx->pkgs;
+				ctx->pkgs = (char *)&meta;
 
 				if (cat == NULL || strcmp(cat->name, atom->CATEGORY) != 0)
 				{
-					if (cat != NULL)
+					if (cat != NULL) {
+						atom_implode((depend_atom *)cat->pkg_ctxs);
+						cat->pkg_ctxs = NULL;
 						tree_close_cat(cat);
-					pkg->cat_ctx = cat = tree_open_cat(ctx, atom->CATEGORY);
+					}
+					pkg.cat_ctx = cat = tree_open_cat(ctx, atom->CATEGORY);
+					cat->pkg_ctxs = (tree_pkg_ctx **)atom;  /* for name */
 				}
 				pkgnamelen = snprintf(pkgname, sizeof(pkgname),
 						"%s.tbz2", atom->PF);
 				pkgname[pkgnamelen - (sizeof(".tbz2") - 1)] = '\0';
-				pkg->name = pkgname;
-				pkg->slot = meta->Q_SLOT == NULL ? (char *)"0" : meta->Q_SLOT;
-				pkg->repo = ctx->repo;
-				pkg->atom = atom;
-				pkg->fd = 0;  /* intentional, meta has already been read */
+				pkg.name = pkgname;
+				pkg.slot = meta.Q_SLOT == NULL ? (char *)"0" : meta.Q_SLOT;
+				pkg.repo = ctx->repo;
+				pkg.atom = atom;
+				pkg.fd = 0;  /* intentional, meta has already been read */
 
 				/* do call callback with pkg_atom (populate cat and pkg) */
-				ret |= callback(pkg, priv);
+				ret |= callback(&pkg, priv);
 
-				atom_implode(atom);
+				ctx->pkgs = c;
+				if (atom != (depend_atom *)cat->pkg_ctxs)
+					atom_implode(atom);
 			}
 
-			memset(meta, 0, sizeof(meta[0]));
+			memset(&meta, 0, sizeof(meta));
 			atom = NULL;
 			if (len > 0) {  /* hop over \n */
 				p++;
@@ -1318,7 +1333,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 #define match_key(X) match_key2(X,X)
 #define match_key2(X,Y) \
 		} else if (strcmp(p, #X) == 0) { \
-			meta->Q_##Y = c
+			meta.Q_##Y = c
 		match_key(DEFINED_PHASES);
 		match_key(DEPEND);
 		match_key2(DESC, DESCRIPTION);
@@ -1339,8 +1354,15 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 		p = q;
 	} while (len > 0);
 
+	if (cat != NULL) {
+		atom_implode((depend_atom *)cat->pkg_ctxs);
+		cat->pkg_ctxs = NULL;
+		tree_close_cat(cat);
+	}
+
 	/* ensure we don't free a garbage pointer */
 	ctx->repo = NULL;
+	ctx->do_sort = false;
 
 	return ret;
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     0467d4c73cc564101ceff32a3b74cc86ea36262c
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 11:48:20 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 11:48:20 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=0467d4c7

libq/tree: Coverity 206569 String not terminated

Help Coverity see the string is terminated, effectively turning a more
expensive zalloc into malloc.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index 1802d79..94f9665 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -793,10 +793,11 @@ tree_read_file_ebuild(tree_pkg_ctx *pkg_ctx)
 		goto err;
 
 	len = sizeof(*ret) + s.st_size + 1;
-	ret = xzalloc(len);
+	ret = xmalloc(len);
 	p = (char *)ret + sizeof(*ret);
 	if ((off_t)fread(p, 1, s.st_size, f) != s.st_size)
 		goto err;
+	p[s.st_size] = '\0';
 
 	do {
 		q = p;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     275298ccf4c33cfe6cc4a816afd8ef15dce6d59f
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 10:04:39 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 10:04:39 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=275298cc

libq/tree: fix initialisation in tree_open_cat

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index f87e751..1802d79 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -262,6 +262,8 @@ tree_open_cat(tree_ctx *ctx, const char *name)
 	cat_ctx->dir = dir;
 	cat_ctx->ctx = ctx;
 	cat_ctx->pkg_ctxs = NULL;
+	cat_ctx->pkg_cur = 0;
+	cat_ctx->pkg_cnt = 0;
 
 	if (ctx->cache.categories != NULL) {
 		add_set_value(name, cat_ctx, ctx->cache.categories);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     3d6b64ef4c4dfd0fe2e008c7cc28a94904775e1e
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 09:44:57 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 09:44:57 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=3d6b64ef

libq/set: change interface of contains_set to return internal key

Allow to refer to the internal allocated key name, which can avoid
another duplicate in certain cases.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/set.c | 11 ++++++-----
 libq/set.h |  3 +--
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/libq/set.c b/libq/set.c
index 4529c3a..ceb6f47 100644
--- a/libq/set.c
+++ b/libq/set.c
@@ -148,23 +148,24 @@ add_set_value(const char *name, void *ptr, set *q)
 	return NULL;
 }
 
-/* returns whether s is in set */
-bool
+/* returns whether name is in set, and if so, the set-internal key
+ * representation (an internal copy of name made during addition) */
+const char *
 contains_set(const char *name, set *q)
 {
 	unsigned int hash;
 	int pos;
 	set_elem *w;
-	bool found;
+	const char *found;
 
 	hash = fnv1a32(name);
 	pos = hash % _SET_HASH_SIZE;
 
-	found = false;
+	found = NULL;
 	if (q->buckets[pos] != NULL) {
 		for (w = q->buckets[pos]; w != NULL; w = w->next) {
 			if (w->hash == hash && strcmp(w->name, name) == 0) {
-				found = true;
+				found = w->name;
 				break;
 			}
 		}

diff --git a/libq/set.h b/libq/set.h
index c65eb0f..5d53f95 100644
--- a/libq/set.h
+++ b/libq/set.h
@@ -7,7 +7,6 @@
 #define _SET_H 1
 
 #include <stdlib.h>
-#include <stdbool.h>
 #include <unistd.h>
 
 #include "xarray.h"
@@ -32,7 +31,7 @@ set *create_set(void);
 set *add_set(const char *name, set *q);
 set *add_set_unique(const char *name, set *q, bool *unique);
 void *add_set_value(const char *name, void *ptr, set *q);
-bool contains_set(const char *name, set *q);
+const char *contains_set(const char *name, set *q);
 void *get_set(const char *name, set *q);
 void *del_set(const char *s, set *q, bool *removed);
 size_t list_set(set *q, char ***l);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     dadb2666f54fed0478e0914d9fc4349f27730d58
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 19 09:46:18 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan 19 09:46:18 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=dadb2666

libq/tree: add initial tree_match_atom with caching

tree_match_atom is meant for retrieving the best matching element from a
tree based on a query.  It caches the underlying packages it traverses,
such that repetitive lookups will benefit from previous lookups.  This
comes in handy for recursive scenarios such as when
calculating/resolving dependencies.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 libq/tree.h |  4 ++++
 2 files changed, 84 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index 0df2e0d..f87e751 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -148,6 +148,24 @@ tree_open_binpkg(const char *sroot, const char *spkg)
 void
 tree_close(tree_ctx *ctx)
 {
+	if (ctx->cache.categories != NULL) {
+		DECLARE_ARRAY(t);
+		size_t n;
+		tree_cat_ctx *cat;
+
+		values_set(ctx->cache.categories, t);
+		free_set(ctx->cache.categories);
+		ctx->cache.categories = NULL;  /* must happen before close_cat */
+
+		array_for_each(t, n, cat) {
+			/* ensure we cleanup all pkgs */
+			cat->pkg_cur = 0;
+			tree_close_cat(cat);
+		}
+
+		xarrayfree_int(t);
+	}
+
 	closedir(ctx->dir);
 	/* closedir() above does this for us: */
 	/* close(ctx->tree_fd); */
@@ -212,6 +230,21 @@ tree_open_cat(tree_ctx *ctx, const char *name)
 	int fd;
 	DIR *dir;
 
+	/* lookup in the cache, if any */
+	if (ctx->cache.categories != NULL) {
+		cat_ctx = get_set(name, ctx->cache.categories);
+		if (cat_ctx != NULL) {
+			/* reset state so it can be re-iterated (sort benefits the
+			 * most here) */
+			if (ctx->do_sort) {
+				cat_ctx->pkg_cur = 0;
+			} else {
+				rewinddir(cat_ctx->dir);
+			}
+			return cat_ctx;
+		}
+	}
+
 	/* Cannot use O_PATH as we want to use fdopendir() */
 	fd = openat(ctx->tree_fd, name, O_RDONLY | O_CLOEXEC);
 	if (fd == -1)
@@ -229,6 +262,13 @@ tree_open_cat(tree_ctx *ctx, const char *name)
 	cat_ctx->dir = dir;
 	cat_ctx->ctx = ctx;
 	cat_ctx->pkg_ctxs = NULL;
+
+	if (ctx->cache.categories != NULL) {
+		add_set_value(name, cat_ctx, ctx->cache.categories);
+		/* ensure name doesn't expire after this instantiation is closed */
+		cat_ctx->name = contains_set(name, ctx->cache.categories);
+	}
+
 	return cat_ctx;
 }
 
@@ -289,6 +329,14 @@ tree_next_cat(tree_ctx *ctx)
 void
 tree_close_cat(tree_cat_ctx *cat_ctx)
 {
+	if (cat_ctx->ctx->cache.categories != NULL &&
+			contains_set(cat_ctx->name, cat_ctx->ctx->cache.categories))
+		return;
+
+	/* cleanup unreturned pkgs when sorted (or cache in use) */
+	while (cat_ctx->pkg_cur < cat_ctx->pkg_cnt)
+		tree_close_pkg(cat_ctx->pkg_ctxs[cat_ctx->pkg_cur++]);
+
 	closedir(cat_ctx->dir);
 	/* closedir() above does this for us: */
 	/* close(ctx->fd); */
@@ -1439,3 +1487,35 @@ tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms)
 
 	return state.cpf;
 }
+
+tree_pkg_ctx *
+tree_match_atom(tree_ctx *ctx, depend_atom *a)
+{
+	tree_cat_ctx *cat_ctx;
+	tree_pkg_ctx *pkg_ctx;
+	depend_atom *atom;
+
+	if (ctx->cache.categories == NULL)
+		ctx->cache.categories = create_set();
+
+	if (a->P == NULL) {
+		return NULL;
+	} else if (a->CATEGORY == NULL) {
+		/* loop through all cats and recurse */
+		/* TODO: some day */
+		return NULL;
+	} else {
+		/* try CAT, and PN for latest version */
+		if ((cat_ctx = tree_open_cat(ctx, a->CATEGORY)) == NULL)
+			return NULL;
+		ctx->do_sort = true;     /* sort uses buffer, which cache relies on */
+		ctx->query_atom = NULL;  /* ensure the cache contains ALL pkgs */
+		while ((pkg_ctx = tree_next_pkg(cat_ctx)) != NULL) {
+			atom = tree_get_atom(pkg_ctx, a->SLOT != NULL);
+			if (atom_compare(atom, a) == EQUAL)
+				return pkg_ctx;
+		}
+
+		return NULL;
+	}
+}

diff --git a/libq/tree.h b/libq/tree.h
index 6627e80..eaee7ad 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -44,6 +44,9 @@ struct tree_ctx {
 	char *pkgs;
 	size_t pkgslen;
 	depend_atom *query_atom;
+	struct tree_cache {
+		set *categories;
+	} cache;
 };
 
 /* Category context */
@@ -135,5 +138,6 @@ int tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 	tree_foreach_pkg(ctx, cb, priv, true, query);
 set *tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms);
 depend_atom *tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete);
+tree_pkg_ctx *tree_match_atom(tree_ctx *t, depend_atom *a);
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     0c691939a77d0056ced7f06d5142c1952f917fee
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Jan 17 08:21:07 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Jan 17 08:21:07 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=0c691939

libq/tree: avoid double free in sorted case for tree_next_pkg_int

Thanks Georgy Yakovlev for the report with stacktrace.

Bug: https://bugs.gentoo.org/705636
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index 0b01f14..0df2e0d 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -403,7 +403,7 @@ tree_next_pkg_int(tree_cat_ctx *cat_ctx)
 				pkg_ctx = cat_ctx->pkg_ctxs[cat_ctx->pkg_cnt++] =
 					tree_open_pkg(cat_ctx, name);
 				if (pkg_ctx == NULL) {
-					free(name);
+					/* name was freed by tree_close_pkg on fail */
 					cat_ctx->pkg_cnt--;
 				}
 			}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     8d4df32d3b04482a072375fd8c4545b4b356de0a
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan  5 15:31:26 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan  5 15:59:27 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=8d4df32d

libq/tree: distinguish between empty and absent file in tree_pkg_meta_get

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 0c05dc5..bb3aa69 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -995,11 +995,12 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 			struct stat s;
 			size_t pos;
 			size_t len;
+			char *p;
 			tree_pkg_meta *m = pkg_ctx->meta;
 
 			if (fd < 0)
 				return NULL;
-			if (fstat(fd, &s) != 0 || s.st_size == 0) {
+			if (fstat(fd, &s) != 0) {
 				close(fd);
 				return NULL;
 			}
@@ -1010,24 +1011,23 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 
 			/* TODO: this is an exact copy from tree_read_file_binpkg_xpak_cb */
 			if (len - pos < (size_t)(s.st_size + 1)) {
-				char *old_data = m->Q__data;
+				p = m->Q__data;
 				len += (((s.st_size + 1) / BUFSIZ) + 1) * BUFSIZ;
 				m->Q__data = xrealloc(m->Q__data, len);
 				m->Q__md5_ = (char *)len;
 
 				/* re-position existing keys */
-				if (old_data != NULL && m->Q__data != old_data) {
+				if (p != NULL && m->Q__data != p) {
 					char **newdata = (char **)m;
 					int elems = sizeof(tree_pkg_meta) / sizeof(char *);
 					while (elems-- > 1)  /* skip Q__data itself */
 						if (newdata[elems] != NULL)
-							newdata[elems] =
-								m->Q__data + (newdata[elems] - old_data);
+							newdata[elems] = m->Q__data + (newdata[elems] - p);
 				}
 			}
 
-			if (read(fd, &m->Q__data[pos], s.st_size) == (ssize_t)s.st_size) {
-				char *p = *key = m->Q__data + pos;
+			p = *key = m->Q__data + pos;
+			if (read(fd, p, s.st_size) == (ssize_t)s.st_size) {
 				p[s.st_size] = '\0';
 				while (s.st_size > 0 && isspace((int)p[s.st_size - 1]))
 					p[--s.st_size] = '\0';

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     a13802923b7fef29333c1af7c2b154a83b5bd739
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan  5 15:48:26 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan  5 15:59:28 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=a1380292

libq/contents: drop expensive checks for newline and tabs

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/contents.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/libq/contents.c b/libq/contents.c
index 3e719a6..7f4351d 100644
--- a/libq/contents.c
+++ b/libq/contents.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2005-2019 Gentoo Foundation
+ * Copyright 2005-2020 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
@@ -24,22 +24,14 @@ contents_parse_line(char *line)
 	static contents_entry e;
 	char *p;
 
-	if (!line || !*line || *line == '\n')
+	if (line == NULL || *line == '\0' || *line == '\n')
 		return NULL;
 
 	/* chop trailing newline */
-	if ((p = strrchr(line, '\n')) != NULL)
+	p = &line[strlen(line) - 1];
+	if (*p == '\n')
 		*p = '\0';
 
-	/* ferringb wants to break portage/vdb by using tabs vs spaces
-	 * so filenames can have lame ass spaces in them..
-	 * (I smell Windows near by)
-	 * Anyway we just convert that crap to a space so we can still
-	 * parse quickly */
-	p = line;
-	while ((p = strchr(p, '\t')) != NULL)
-		*p = ' ';
-
 	memset(&e, 0x00, sizeof(e));
 	e._data = line;
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     98b3632c7e28b4f7d5eebb7b7ffc255adabe81b7
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jan  5 16:08:00 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jan  5 16:08:00 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=98b3632c

libq/tree: make some unused functions private (static)

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 8 ++++----
 libq/tree.h | 3 ---
 2 files changed, 4 insertions(+), 7 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index bb3aa69..0b01f14 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -27,6 +27,8 @@
 #include <xalloc.h>
 
 static int tree_pkg_compar(const void *l, const void *r);
+static tree_pkg_ctx * tree_next_pkg_int(tree_cat_ctx *cat_ctx);
+static void tree_close_meta(tree_pkg_meta *cache);
 
 static tree_ctx *
 tree_open_int(const char *sroot, const char *tdir, bool quiet)
@@ -363,8 +365,6 @@ tree_pkg_compar(const void *l, const void *r)
 	return atom_compar_cb(al, ar);
 }
 
-static tree_pkg_ctx *
-tree_next_pkg_int(tree_cat_ctx *cat_ctx);
 static tree_pkg_ctx *
 tree_next_pkg_int(tree_cat_ctx *cat_ctx)
 {
@@ -926,7 +926,7 @@ tree_read_file_binpkg(tree_pkg_ctx *pkg_ctx)
 	return m;
 }
 
-tree_pkg_meta *
+static tree_pkg_meta *
 tree_pkg_read(tree_pkg_ctx *pkg_ctx)
 {
 	tree_ctx *ctx = pkg_ctx->cat_ctx->ctx;
@@ -963,7 +963,7 @@ tree_pkg_read(tree_pkg_ctx *pkg_ctx)
 	return NULL;
 }
 
-void
+static void
 tree_close_meta(tree_pkg_meta *cache)
 {
 	if (cache == NULL)

diff --git a/libq/tree.h b/libq/tree.h
index c941172..6627e80 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -121,8 +121,6 @@ tree_cat_ctx *tree_open_cat(tree_ctx *ctx, const char *name);
 void tree_close_cat(tree_cat_ctx *cat_ctx);
 tree_pkg_ctx *tree_open_pkg(tree_cat_ctx *cat_ctx, const char *name);
 tree_pkg_ctx *tree_next_pkg(tree_cat_ctx *cat_ctx);
-tree_pkg_meta *tree_pkg_read(tree_pkg_ctx *pkg_ctx);
-void tree_close_meta(tree_pkg_meta *cache);
 char *tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *key);
 #define tree_pkg_meta_get(P,X) \
 	tree_pkg_meta_get_int(P, offsetof(tree_pkg_meta, Q_##X), #X)
@@ -135,7 +133,6 @@ int tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 	tree_foreach_pkg(ctx, cb, priv, false, query);
 #define tree_foreach_pkg_sorted(ctx, cb, priv, query) \
 	tree_foreach_pkg(ctx, cb, priv, true, query);
-struct dirent *tree_get_next_dir(DIR *dir);
 set *tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms);
 depend_atom *tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete);
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     bab8be1e6ef9998ca7285310a4fd135127442500
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Jan  2 15:09:21 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Jan  2 15:09:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=bab8be1e

libq/tree: ensure we don't leak scandir results on sorted foreach_pkg

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index 186fb4d..49b2fa1 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1341,9 +1341,12 @@ tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 	}
 
 	/* allow foreach to be called again on the same open tree */
+	if (ctx->do_sort)
+		scandir_free(ctx->cat_de, ctx->cat_cnt);
 	ctx->cat_de = NULL;
 	ctx->cat_cur = 0;
 	ctx->cat_cnt = 0;
+	ctx->do_sort = 0;
 
 	return ret;
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     70676c12f78bcf99ebc29901a6db36cbee6044c9
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Jan  2 14:04:19 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Jan  2 14:04:19 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=70676c12

libq/tree: fix off-by-ones

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index e7914f1..186fb4d 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -912,7 +912,7 @@ tree_read_file_binpkg_xpak_cb(
 	while (isspace((int)data[data_offset + data_len - 1]))
 		data_len--;
 
-	if (len - pos < (size_t)data_len) {
+	if (len - pos < (size_t)(data_len + 1)) {
 		char *old_data = m->Q__data;
 		len += (((data_len + 1) / BUFSIZ) + 1) * BUFSIZ;
 		m->Q__data = xrealloc(m->Q__data, len);
@@ -1028,7 +1028,7 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 			len = (size_t)m->Q__md5_;
 
 			/* TODO: this is an exact copy from tree_read_file_binpkg_xpak_cb */
-			if (len - pos < (size_t)s.st_size) {
+			if (len - pos < (size_t)(s.st_size + 1)) {
 				char *old_data = m->Q__data;
 				len += (((s.st_size + 1) / BUFSIZ) + 1) * BUFSIZ;
 				m->Q__data = xrealloc(m->Q__data, len);
@@ -1049,7 +1049,7 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 				char *p = *key = m->Q__data + pos;
 				p[s.st_size] = '\0';
 				while (s.st_size > 0 && isspace((int)p[s.st_size - 1]))
-					p[s.st_size--] = '\0';
+					p[--s.st_size] = '\0';
 				pos += s.st_size + 1;
 				m->Q__eclasses_ = (char *)pos;
 			}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e575a3259cc0d74972359315f549f27372e1f6b6
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Jan  2 13:22:22 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Jan  2 13:22:22 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e575a325

libq/tree: fix bug after realloc in metadata Q__data

don't reposition the Q__data pointer itself, it was already updated to
the new value

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 570859e..e7914f1 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -922,7 +922,7 @@ tree_read_file_binpkg_xpak_cb(
 		if (old_data != NULL && m->Q__data != old_data) {
 			char **newdata = (char **)m;
 			int elems = sizeof(tree_pkg_meta) / sizeof(char *);
-			while (elems-- > 0)
+			while (elems-- > 1)  /* skip Q__data itself */
 				if (newdata[elems] != NULL)
 					newdata[elems] = m->Q__data + (newdata[elems] - old_data);
 		}
@@ -999,7 +999,8 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 	char **key;
 
 	/* offset is a byte offset in the tree_pkg_meta struct, pointing to
-	 * key, the tree_pkg_meta_get macro takes care of this */
+	 * key, the tree_pkg_meta_get macro as called by the user takes care
+	 * of offset and keyn pointing to the same thing */
 
 	if (ctx->cachetype == CACHE_VDB) {
 		if (pkg_ctx->meta == NULL)
@@ -1037,7 +1038,7 @@ tree_pkg_meta_get_int(tree_pkg_ctx *pkg_ctx, size_t offset, const char *keyn)
 				if (old_data != NULL && m->Q__data != old_data) {
 					char **newdata = (char **)m;
 					int elems = sizeof(tree_pkg_meta) / sizeof(char *);
-					while (elems-- > 0)
+					while (elems-- > 1)  /* skip Q__data itself */
 						if (newdata[elems] != NULL)
 							newdata[elems] =
 								m->Q__data + (newdata[elems] - old_data);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     ad34656e2d3cd6400c6debf3c46f39ab19219a6b
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Jan  2 12:45:52 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Jan  2 12:45:52 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ad34656e

libq/tree: allow tree_foreach_pkg to be called multiple times

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index 0fdf9b1..570859e 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1339,6 +1339,11 @@ tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 		tree_close_cat(cat_ctx);
 	}
 
+	/* allow foreach to be called again on the same open tree */
+	ctx->cat_de = NULL;
+	ctx->cat_cur = 0;
+	ctx->cat_cnt = 0;
+
 	return ret;
 }
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     74a6ea62eb1b9c9ab4c6ed3402e9c7d7e618ceb4
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Jan  2 11:43:10 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Jan  2 11:43:10 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=74a6ea62

libq/tree: activate atom query

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 8996e55..0fdf9b1 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1192,8 +1192,7 @@ tree_close_pkg(tree_pkg_ctx *pkg_ctx)
 }
 
 static int
-tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback,
-		void *priv, depend_atom *query)
+tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 {
 	char *p = ctx->pkgs;
 	char *q;
@@ -1201,6 +1200,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback,
 	char pkgname[_Q_PATH_MAX];
 	size_t len = ctx->pkgslen;
 	int ret = 0;
+	depend_atom *query = ctx->query_atom;
 
 	/* reused for every entry */
 	tree_cat_ctx *cat = NULL;
@@ -1323,11 +1323,12 @@ tree_foreach_pkg(tree_ctx *ctx, tree_pkg_cb callback, void *priv,
 	if (ctx == NULL)
 		return EXIT_FAILURE;
 
+	ctx->do_sort = sort;
+	ctx->query_atom = query;
+
 	/* handle Packages (binpkgs index) file separately */
 	if (ctx->cachetype == CACHE_PACKAGES)
-		return tree_foreach_packages(ctx, callback, priv, query);
-
-	ctx->do_sort = sort;
+		return tree_foreach_packages(ctx, callback, priv);
 
 	ret = 0;
 	while ((cat_ctx = tree_next_cat(ctx))) {

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     f177ab6944e899a8444640ed125ee7f4e1f6589d
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Jan  2 10:32:27 2020 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Jan  2 10:32:27 2020 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=f177ab69

libq/atom: also strip/ignore .tbz2 suffix (like .ebuild)

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libq/atom.c b/libq/atom.c
index 946e821..e79c30a 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -118,7 +118,8 @@ atom_explode(const char *atom)
 	strcpy(ret->CATEGORY, atom);
 
 	/* eat file name crap when given an (autocompleted) path */
-	if ((ptr = strstr(ret->CATEGORY, ".ebuild")) != NULL)
+	if ((ptr = strstr(ret->CATEGORY, ".ebuild")) != NULL ||
+			(ptr = strstr(ret->CATEGORY, ".tbz2")) != NULL)
 		*ptr = '\0';
 
 	/* chip off the trailing ::REPO as needed */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     c1a7c3366024334c97303cad6e1e54e30285beda
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Dec 30 09:16:03 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Dec 30 09:16:03 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=c1a7c336

libq/tree: ensure valid pointers in case of realloc

in tree_read_file_binpkg_xpak_cb pointers could get stale after realloc,
if so, reposition them in the newly allocated block

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index f0c8ddb..d9eec76 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -876,9 +876,19 @@ tree_read_file_binpkg_xpak_cb(
 		data_len--;
 
 	if (len - pos < (size_t)data_len) {
+		char *old_data = m->_data;
 		len += (((data_len + 1) / BUFSIZ) + 1) * BUFSIZ;
 		m->_data = xrealloc(m->_data, len);
 		m->_md5_ = (char *)len;
+
+		/* re-position existing keys */
+		if (old_data != NULL && m->_data != old_data) {
+			char **newdata = (char **)m;
+			int elems = sizeof(tree_pkg_meta) / sizeof(char *);
+			while (elems-- > 0)
+				if (newdata[elems] != NULL)
+					newdata[elems] = m->_data + (newdata[elems] - old_data);
+		}
 	}
 
 	*key = m->_data + pos;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e2b0611a5d20eb60a5b5362975703b1909704289
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Dec 27 21:18:46 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Dec 27 21:18:46 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e2b0611a

libq/atom: fix crash in atom_compare on invalid input

Bug: https://bugs.gentoo.org/700850
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/atom.c b/libq/atom.c
index efd32d1..6f65c8b 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -525,7 +525,7 @@ atom_compare(const depend_atom *data, const depend_atom *query)
 	if (sfx_op == ATOM_OP_STAR) {
 		if (query->letter)
 			ver_bits |= (1 << 0);
-		if (query->suffixes[0].suffix != VER_NORM)
+		if (query->suffixes && query->suffixes[0].suffix != VER_NORM)
 			ver_bits |= (1 << 1);
 		/* This doesn't handle things like foo-1.0-r0*, but that atom
 		 * doesn't ever show up in practice, so who cares. */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     ea01f98af2c34f59b762f83f4a45717b53d9d635
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Dec 27 14:34:22 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Dec 27 14:34:22 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ea01f98a

libq/set: rename elem to set_elem

elem is a tad bit too generic a name

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/set.c | 30 +++++++++++++++---------------
 libq/set.h |  6 +++---
 2 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/libq/set.c b/libq/set.c
index 3b56f81..28c41a1 100644
--- a/libq/set.c
+++ b/libq/set.c
@@ -38,8 +38,8 @@ set *
 add_set(const char *name, set *q)
 {
 	int pos;
-	elem *ll = xmalloc(sizeof(*ll));
-	elem *w;
+	set_elem *ll = xmalloc(sizeof(*ll));
+	set_elem *w;
 
 	if (q == NULL)
 		q = create_set();
@@ -68,8 +68,8 @@ add_set_unique(const char *name, set *q, bool *unique)
 {
 	unsigned int hash;
 	int pos;
-	elem *ll;
-	elem *w;
+	set_elem *ll;
+	set_elem *w;
 	bool uniq = false;
 
 	if (q == NULL)
@@ -117,8 +117,8 @@ add_set_value(const char *name, void *ptr, set *q)
 {
 	unsigned int hash;
 	int pos;
-	elem *ll;
-	elem *w;
+	set_elem *ll;
+	set_elem *w;
 
 	hash = fnv1a32(name);
 	pos = hash % _SET_HASH_SIZE;
@@ -154,7 +154,7 @@ contains_set(const char *name, set *q)
 {
 	unsigned int hash;
 	int pos;
-	elem *w;
+	set_elem *w;
 	bool found;
 
 	hash = fnv1a32(name);
@@ -180,7 +180,7 @@ get_set(const char *name, set *q)
 {
 	unsigned int hash;
 	int pos;
-	elem *w;
+	set_elem *w;
 
 	hash = fnv1a32(name);
 	pos = hash % _SET_HASH_SIZE;
@@ -205,8 +205,8 @@ del_set(const char *s, set *q, bool *removed)
 {
 	unsigned int hash;
 	int pos;
-	elem *ll;
-	elem *w;
+	set_elem *ll;
+	set_elem *w;
 	void *ret;
 	bool rmd;
 
@@ -248,7 +248,7 @@ size_t
 list_set(set *q, char ***l)
 {
 	int i;
-	elem *w;
+	set_elem *w;
 	char **ret;
 
 	ret = *l = xmalloc(sizeof(char **) * (q->len + 1));
@@ -266,7 +266,7 @@ size_t
 values_set(set *q, array_t *ret)
 {
 	int i;
-	elem *w;
+	set_elem *w;
 	array_t blank = array_init_decl;
 
 	*ret = blank;
@@ -289,8 +289,8 @@ void
 clear_set(set *q)
 {
 	int i;
-	elem *w;
-	elem *e;
+	set_elem *w;
+	set_elem *e;
 
 	for (i = 0; i < _SET_HASH_SIZE; i++) {
 		for (w = q->buckets[i]; w != NULL; w = e) {
@@ -315,7 +315,7 @@ free_set(set *q)
 static void
 print_set(const set *q)
 {
-	elem *w;
+	set_elem *w;
 	int i;
 
 	for (i = 0; i < _SET_HASH_SIZE; i++) {

diff --git a/libq/set.h b/libq/set.h
index 8546a90..118d20d 100644
--- a/libq/set.h
+++ b/libq/set.h
@@ -12,19 +12,19 @@
 
 #include "xarray.h"
 
-typedef struct elem_t elem;
+typedef struct elem_t set_elem;
 typedef struct set_t set;
 
 struct elem_t {
 	char *name;
 	unsigned int hash;  /* FNV1a32 */
 	void *val;
-	elem *next;
+	set_elem *next;
 };
 
 #define _SET_HASH_SIZE 128
 struct set_t {
-	elem *buckets[_SET_HASH_SIZE];
+	set_elem *buckets[_SET_HASH_SIZE];
 	size_t len;
 };
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     a5dc7cb75f0dc1edd85ee26ce9d1a6636d4b40c0
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Dec 27 14:56:12 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Dec 27 14:56:12 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=a5dc7cb7

libq/set: add array_set function

like list_set, array_set returns all keys, but in an libq/xarray instead
of C-array.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/set.c | 16 ++++++++++++++++
 libq/set.h |  1 +
 2 files changed, 17 insertions(+)

diff --git a/libq/set.c b/libq/set.c
index 28c41a1..4529c3a 100644
--- a/libq/set.c
+++ b/libq/set.c
@@ -262,6 +262,22 @@ list_set(set *q, char ***l)
 	return q->len;
 }
 
+size_t
+array_set(set *q, array_t *ret)
+{
+	int i;
+	set_elem *w;
+	array_t blank = array_init_decl;
+
+	*ret = blank;
+	for (i = 0; i < _SET_HASH_SIZE; i++) {
+		for (w = q->buckets[i]; w != NULL; w = w->next)
+			xarraypush_ptr(ret, w->name);
+	}
+
+	return q->len;
+}
+
 size_t
 values_set(set *q, array_t *ret)
 {

diff --git a/libq/set.h b/libq/set.h
index 118d20d..c65eb0f 100644
--- a/libq/set.h
+++ b/libq/set.h
@@ -36,6 +36,7 @@ bool contains_set(const char *name, set *q);
 void *get_set(const char *name, set *q);
 void *del_set(const char *s, set *q, bool *removed);
 size_t list_set(set *q, char ***l);
+size_t array_set(set *q, array_t *ret);
 size_t values_set(set *q, array_t *ret);
 size_t cnt_set(set *q);
 void free_set(set *q);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     50b70d7c6696903440adde3a945af8bf298e1a0b
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Nov 29 13:18:37 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Nov 29 13:18:37 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=50b70d7c

libq/tree: fix crash in tree_close_meta, bug #701470

_data member was used differently before, make sure we can free it

Bug: https://bugs.gentoo.org/701470
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 19 +++++++------------
 1 file changed, 7 insertions(+), 12 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index a383950..cc8cf3e 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -550,12 +550,11 @@ tree_read_file_pms(tree_pkg_ctx *pkg_ctx)
 
 	len = sizeof(*ret) + s.st_size + 1;
 	ret = xzalloc(len);
-	ptr = (char*)ret;
-	ret->_data = ptr + sizeof(*ret);
-	if ((off_t)fread(ret->_data, 1, s.st_size, f) != s.st_size)
+	ptr = (char*)ret + sizeof(*ret);
+	if ((off_t)fread(ptr, 1, s.st_size, f) != s.st_size)
 		goto err;
 
-	ret->DEPEND = ret->_data;
+	ret->DEPEND = ptr;
 #define next_line(curr, next) \
 	if ((ptr = strchr(ret->curr, '\n')) == NULL) { \
 		warn("Invalid cache file for '%s'", buf); \
@@ -618,9 +617,8 @@ tree_read_file_md5(tree_pkg_ctx *pkg_ctx)
 
 	len = sizeof(*ret) + s.st_size + 1;
 	ret = xzalloc(len);
-	ptr = (char*)ret;
-	ret->_data = ptr + sizeof(*ret);
-	if ((off_t)fread(ret->_data, 1, s.st_size, f) != s.st_size)
+	ptr = (char*)ret + sizeof(*ret);
+	if ((off_t)fread(ptr, 1, s.st_size, f) != s.st_size)
 		goto err;
 
 	/* We have a block of key=value\n data.
@@ -638,7 +636,6 @@ tree_read_file_md5(tree_pkg_ctx *pkg_ctx)
 #define assign_var(keyname) \
 	assign_var_cmp(keyname, #keyname);
 
-	ptr = ret->_data;
 	endptr = strchr(ptr, '\0');
 	if (endptr == NULL) {
 			warn("Invalid cache file for '%s/%s': "
@@ -731,12 +728,10 @@ tree_read_file_ebuild(tree_pkg_ctx *pkg_ctx)
 
 	len = sizeof(*ret) + s.st_size + 1;
 	ret = xzalloc(len);
-	p = (char *)ret;
-	ret->_data = p + sizeof(*ret);
-	if ((off_t)fread(ret->_data, 1, s.st_size, f) != s.st_size)
+	p = (char *)ret + sizeof(*ret);
+	if ((off_t)fread(p, 1, s.st_size, f) != s.st_size)
 		goto err;
 
-	p = ret->_data;
 	do {
 		q = p;
 		while (*p >= 'A' && *p <= 'Z')

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     5574f98fd76030581c75622bcb7f2f2c85cad234
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Nov 20 17:22:29 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Nov 20 17:22:29 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=5574f98f

libq/xpak: fix out of bounds checking

Don't check bounds if there is no data retrieved, and make sure we check
the bounds on the data length, not index length.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c |  2 +-
 libq/xpak.c | 11 ++++++-----
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 427281f..a383950 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1094,7 +1094,7 @@ tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
 
 				cat->name = atom->CATEGORY;
 				pkg->name = atom->PN;
-				pkg->slot = meta->SLOT == NULL ? "0" : meta->SLOT;
+				pkg->slot = meta->SLOT == NULL ? (char *)"0" : meta->SLOT;
 				pkg->repo = ctx->repo;
 				pkg->atom = atom;
 

diff --git a/libq/xpak.c b/libq/xpak.c
index 71dc17e..90a3570 100644
--- a/libq/xpak.c
+++ b/libq/xpak.c
@@ -79,11 +79,12 @@ static void _xpak_walk_index(
 		p += 4;
 
 		/* check offset and len individually to deal with overflow */
-		if (data_offset > x->index_len ||
-				data_len > x->index_len ||
-				data_offset + data_len > x->index_len)
-			err("Data for '%s' is out of bounds: offset=%u, len=%u\n",
-					pathname, data_len, data_offset);
+		if (x->data != NULL &&
+				(data_offset > x->data_len ||
+				 data_len > x->data_len ||
+				 data_offset + data_len > x->data_len))
+			err("Data for '%s' is out of bounds: offset=%u, len=%u, size=%u\n",
+					pathname, data_len, data_offset, x->data_len);
 
 		(*func)(x->ctx, pathname, pathname_len,
 				data_offset, data_len, x->data);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     3763b1135eefc63fa19c084602241767670a4dd0
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Nov 18 20:07:08 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Nov 18 20:07:08 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=3763b113

libq/xpak: perform sanity checks on offset and len

As shown by Agostino Sarubbo, the input can be crap, resulting in very
bad scenarios.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xpak.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/libq/xpak.c b/libq/xpak.c
index 82667b0..71dc17e 100644
--- a/libq/xpak.c
+++ b/libq/xpak.c
@@ -77,6 +77,14 @@ static void _xpak_walk_index(
 		p += 4;
 		data_len = READ_BE_INT32((unsigned char*)p);
 		p += 4;
+
+		/* check offset and len individually to deal with overflow */
+		if (data_offset > x->index_len ||
+				data_len > x->index_len ||
+				data_offset + data_len > x->index_len)
+			err("Data for '%s' is out of bounds: offset=%u, len=%u\n",
+					pathname, data_len, data_offset);
+
 		(*func)(x->ctx, pathname, pathname_len,
 				data_offset, data_len, x->data);
 	}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     aeadc86f00479f0b842c8f3f0905d21b35565c13
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Nov 17 12:46:22 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Nov 17 12:46:22 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=aeadc86f

libq/xpak: try to seek to XPAKSTART in _xpak_open

To make it easier to deal with assembled files (like binpkgs) just embed
the offset logic in _xpak_open.  This makes qxpak operate directly on
tbz2 files, removing the need to split out using qtbz2 first.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xpak.c | 39 +++++++++++++++++++++++++++++++++------
 1 file changed, 33 insertions(+), 6 deletions(-)

diff --git a/libq/xpak.c b/libq/xpak.c
index 9692eab..82667b0 100644
--- a/libq/xpak.c
+++ b/libq/xpak.c
@@ -4,6 +4,7 @@
  *
  * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
  * Copyright 2005-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2019-     Fabian Groffen  - <grobian@gentoo.org>
  */
 
 #include "main.h"
@@ -37,6 +38,10 @@
 #define XPAK_START_LEN       (8 + 4 + 4)
 #define XPAK_END_MSG         "XPAKSTOP"
 #define XPAK_END_MSG_LEN     8
+#define TBZ2_END_MSG         "STOP"
+#define TBZ2_END_MSG_LEN     4
+#define TBZ2_END_SIZE_LEN    4
+#define TBZ2_FOOTER_LEN      (TBZ2_END_MSG_LEN + TBZ2_END_SIZE_LEN)
 
 typedef struct {
 	void *ctx;
@@ -87,14 +92,36 @@ static _xpak_archive *_xpak_open(const int fd)
 	if ((ret.fp = fdopen(fd, "r")) == NULL)
 		return NULL;
 
-	/* verify this xpak doesnt suck */
+	/* verify this xpak doesn't suck */
 	if (fread(buf, 1, XPAK_START_LEN, ret.fp) != XPAK_START_LEN)
 		goto close_and_ret;
-	if (memcmp(buf, XPAK_START_MSG, XPAK_START_MSG_LEN)) {
+	if (memcmp(buf, XPAK_START_MSG, XPAK_START_MSG_LEN) != 0) {
+		/* stream not positioned at XPAKSTART, let's see if we can
+		 * reposition the stream */
+		if (fseek(ret.fp, -TBZ2_FOOTER_LEN, SEEK_END) == 0)
+		{
+			if (fread(buf, 1, TBZ2_FOOTER_LEN, ret.fp) != TBZ2_FOOTER_LEN)
+				goto close_and_ret;
+
+			if (memcmp(buf + TBZ2_END_SIZE_LEN,
+						TBZ2_END_MSG, TBZ2_END_MSG_LEN) == 0)
+			{
+				int xpaklen = READ_BE_INT32(buf);
+
+				if (fseek(ret.fp, -(xpaklen + TBZ2_FOOTER_LEN), SEEK_END) == 0)
+				{
+					if (fread(buf, 1, XPAK_START_LEN, ret.fp) != XPAK_START_LEN)
+						goto close_and_ret;
+					if (memcmp(buf, XPAK_START_MSG, XPAK_START_MSG_LEN) == 0)
+						goto setup_lens;
+				}
+			}
+		}
 		warn("Not an xpak file");
 		goto close_and_ret;
 	}
 
+setup_lens:
 	/* calc index and data sizes */
 	ret.index_len = READ_BE_INT32((unsigned char*)buf+XPAK_START_MSG_LEN);
 	ret.data_len = READ_BE_INT32((unsigned char*)buf+XPAK_START_MSG_LEN+4);
@@ -124,7 +151,7 @@ xpak_process_fd(
 	xpak_callback_t func)
 {
 	_xpak_archive *x;
-	char buf[BUFSIZE], ext[BUFSIZE*32];
+	char buf[BUFSIZE];
 	size_t in;
 
 	x = _xpak_open(fd);
@@ -142,8 +169,8 @@ xpak_process_fd(
 
 	if (get_data) {
 		/* the xpak may be large (like when it has CONTENTS) #300744 */
-		x->data = (size_t)x->data_len < sizeof(ext) ?
-			ext : xmalloc(x->data_len);
+		x->data = xmalloc(x->data_len);
+
 		in = fread(x->data, 1, x->data_len, x->fp);
 		if (in != (size_t)x->data_len)
 			err("insufficient data read, got %zd, requested %d",
@@ -157,7 +184,7 @@ xpak_process_fd(
 
 	_xpak_close(x);
 
-	if (get_data && x->data != ext)
+	if (get_data)
 		free(x->data);
 
 	return 0;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     b811b9afd740e8f4cc73f096e8c162d5d332aed2
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Nov 17 15:01:00 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Nov 17 15:01:00 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=b811b9af

libq/tree: add support for binpgks and Packages file

This allows to foreach binpkgs (without Packages file index) as well as
use the Packages file to loop over them.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 254 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++----
 libq/tree.h |  13 +++-
 2 files changed, 251 insertions(+), 16 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index b0d0b5b..427281f 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -21,6 +21,7 @@
 #include "scandirat.h"
 #include "set.h"
 #include "tree.h"
+#include "xpak.h"
 
 #include <ctype.h>
 #include <xalloc.h>
@@ -30,7 +31,7 @@ static int tree_pkg_compar(const void *l, const void *r);
 static tree_ctx *
 tree_open_int(const char *sroot, const char *tdir, bool quiet)
 {
-	tree_ctx *ctx = xmalloc(sizeof(*ctx));
+	tree_ctx *ctx = xzalloc(sizeof(*ctx));
 
 	ctx->portroot_fd = open(sroot, O_RDONLY | O_CLOEXEC | O_PATH);
 	if (ctx->portroot_fd == -1) {
@@ -56,12 +57,8 @@ tree_open_int(const char *sroot, const char *tdir, bool quiet)
 		goto cv_error;
 
 	ctx->do_sort = false;
-	ctx->cat_de = NULL;
 	ctx->catsortfunc = alphasort;
 	ctx->pkgsortfunc = tree_pkg_compar;
-	ctx->repo = NULL;
-	ctx->ebuilddir_ctx = NULL;
-	ctx->ebuilddir_pkg_ctx = NULL;
 	return ctx;
 
  cv_error:
@@ -130,6 +127,24 @@ tree_open_vdb(const char *sroot, const char *svdb)
 	return ret;
 }
 
+static const char binpkg_packages[]  = "Packages";
+tree_ctx *
+tree_open_binpkg(const char *sroot, const char *spkg)
+{
+	tree_ctx *ret = tree_open_int(sroot, spkg, true);
+	char buf[_Q_PATH_MAX];
+
+	if (ret != NULL) {
+		ret->cachetype = CACHE_BINPKGS;
+
+		snprintf(buf, sizeof(buf), "%s%s/%s", sroot, spkg, binpkg_packages);
+		if (eat_file(buf, &ret->pkgs, &ret->pkgslen))
+			ret->cachetype = CACHE_PACKAGES;
+	}
+
+	return ret;
+}
+
 void
 tree_close(tree_ctx *ctx)
 {
@@ -141,6 +156,8 @@ tree_close(tree_ctx *ctx)
 		scandir_free(ctx->cat_de, ctx->cat_cnt);
 	if (ctx->repo != NULL)
 		free(ctx->repo);
+	if (ctx->pkgs != NULL)
+		free(ctx->pkgs);
 	if (ctx->ebuilddir_ctx != NULL)
 		free(ctx->ebuilddir_ctx);
 	free(ctx);
@@ -443,6 +460,13 @@ tree_next_pkg(tree_cat_ctx *cat_ctx)
 				}
 			}
 		} while (ret == NULL);
+	} else if (ctx->cachetype == CACHE_BINPKGS) {
+		char *p = NULL;
+		do {
+			ret = tree_next_pkg_int(cat_ctx);
+		} while (ret != NULL && (p = strstr(ret->name, ".tbz2")) == NULL);
+		if (p != NULL)
+			*p = '\0';
 	} else {
 		ret = tree_next_pkg_int(cat_ctx);
 	}
@@ -805,22 +829,96 @@ err:
 	return NULL;
 }
 
+static void
+tree_read_file_binpkg_xpak_cb(
+	void *ctx,
+	char *pathname,
+	int pathname_len,
+	int data_offset,
+	int data_len,
+	char *data)
+{
+	tree_pkg_meta *m = (tree_pkg_meta *)ctx;
+	char **key;
+	size_t pos;
+	size_t len;
+
+#define match_path(K) \
+	else if (pathname_len == (sizeof(#K) - 1) && strcmp(pathname, #K) == 0) \
+		key = &m->K
+	if (1 == 0); /* dummy for syntax */
+	match_path(DEPEND);
+	match_path(RDEPEND);
+	match_path(SLOT);
+	match_path(SRC_URI);
+	match_path(RESTRICT);
+	match_path(HOMEPAGE);
+	match_path(DESCRIPTION);
+	match_path(KEYWORDS);
+	match_path(INHERITED);
+	match_path(IUSE);
+	match_path(CDEPEND);
+	match_path(PDEPEND);
+	match_path(PROVIDE);
+	match_path(EAPI);
+	match_path(PROPERTIES);
+	match_path(DEFINED_PHASES);
+	match_path(REQUIRED_USE);
+	match_path(BDEPEND);
+	match_path(CONTENTS);
+	match_path(USE);
+	match_path(repository);
+	else
+		return;
+#undef match_path
+
+	/* hijack unused members */
+	pos = (size_t)m->_eclasses_;
+	len = (size_t)m->_md5_;
+
+	/* trim whitespace (mostly trailing newline) */
+	while (isspace((int)data[data_offset + data_len - 1]))
+		data_len--;
+
+	if (len - pos < (size_t)data_len) {
+		len += (((data_len + 1) / BUFSIZ) + 1) * BUFSIZ;
+		m->_data = xrealloc(m->_data, len);
+		m->_md5_ = (char *)len;
+	}
+
+	*key = m->_data + pos;
+	snprintf(*key, len - pos, "%.*s", data_len, data + data_offset);
+	pos += data_len + 1;
+	m->_eclasses_ = (char *)pos;
+}
+
+static tree_pkg_meta *
+tree_read_file_binpkg(tree_pkg_ctx *pkg_ctx)
+{
+	tree_pkg_meta *m = xzalloc(sizeof(tree_pkg_meta));
+
+	xpak_process_fd(pkg_ctx->fd, true, m, tree_read_file_binpkg_xpak_cb);
+	pkg_ctx->fd = -1;  /* closed by xpak_process_fd */
+
+	return m;
+}
+
 tree_pkg_meta *
 tree_pkg_read(tree_pkg_ctx *pkg_ctx)
 {
 	tree_ctx *ctx = pkg_ctx->cat_ctx->ctx;
 
 	if (pkg_ctx->fd == -1) {
-		if (ctx->cachetype != CACHE_EBUILD) {
-			pkg_ctx->fd = openat(pkg_ctx->cat_ctx->fd, pkg_ctx->name,
-					O_RDONLY | O_CLOEXEC);
-		} else {
+		if (ctx->cachetype == CACHE_EBUILD || ctx->cachetype == CACHE_BINPKGS) {
 			char *p = (char *)pkg_ctx->name;
 			p += strlen(p);
 			*p = '.';
 			pkg_ctx->fd = openat(pkg_ctx->cat_ctx->fd, pkg_ctx->name,
 					O_RDONLY | O_CLOEXEC);
 			*p = '\0';
+		} else {
+			pkg_ctx->fd = openat(pkg_ctx->cat_ctx->fd, pkg_ctx->name,
+					O_RDONLY | O_CLOEXEC);
 		}
 		if (pkg_ctx->fd == -1)
 			return NULL;
@@ -832,6 +930,10 @@ tree_pkg_read(tree_pkg_ctx *pkg_ctx)
 		return tree_read_file_pms(pkg_ctx);
 	} else if (ctx->cachetype == CACHE_EBUILD) {
 		return tree_read_file_ebuild(pkg_ctx);
+	} else if (ctx->cachetype == CACHE_BINPKGS) {
+		return tree_read_file_binpkg(pkg_ctx);
+	} else if (ctx->cachetype == CACHE_PACKAGES) {
+		return (tree_pkg_meta *)pkg_ctx->cat_ctx->ctx->pkgs;
 	}
 
 	warn("Unknown metadata cache type!");
@@ -841,8 +943,10 @@ tree_pkg_read(tree_pkg_ctx *pkg_ctx)
 void
 tree_close_meta(tree_pkg_meta *cache)
 {
-	if (!cache)
+	if (cache == NULL)
 		errf("Cache is empty !");
+	if (cache->_data != NULL)
+		free(cache->_data);
 	free(cache);
 }
 
@@ -952,6 +1056,112 @@ tree_close_pkg(tree_pkg_ctx *pkg_ctx)
 	free(pkg_ctx);
 }
 
+static int
+tree_foreach_packages(tree_ctx *ctx, tree_pkg_cb callback, void *priv)
+{
+	char *p = ctx->pkgs;
+	char *q;
+	char *c;
+	size_t len = ctx->pkgslen;
+	int ret = 0;
+
+	/* reused for every entry */
+	tree_cat_ctx *cat = xzalloc(sizeof(tree_cat_ctx));
+	tree_pkg_ctx *pkg = xzalloc(sizeof(tree_pkg_ctx));
+	tree_pkg_meta *meta = xzalloc(sizeof(tree_pkg_meta));
+	depend_atom *atom = NULL;
+
+	cat->ctx = ctx;
+	pkg->cat_ctx = cat;
+
+	do {
+		/* find next line */
+		c = NULL;
+		for (q = p; len > 0 && *q != '\n'; q++, len--)
+			if (c == NULL && *q == ':')
+				c = q;
+
+		if (len == 0)
+			break;
+
+		/* empty line, end of a block */
+		if (p == q) {
+			/* make callback with populated atom */
+			if (atom != NULL) {
+				/* store meta ptr in repo->pkgs, such that get_pkg_meta
+				 * can grab it from there (for free) */
+				ctx->pkgs = (char *)meta;
+
+				cat->name = atom->CATEGORY;
+				pkg->name = atom->PN;
+				pkg->slot = meta->SLOT == NULL ? "0" : meta->SLOT;
+				pkg->repo = ctx->repo;
+				pkg->atom = atom;
+
+				/* do call callback with pkg_atom (populate cat and pkg) */
+				ret |= callback(pkg, priv);
+
+				atom_implode(atom);
+			}
+
+			memset(meta, 0, sizeof(meta[0]));
+			atom = NULL;
+			if (len > 0) {  /* hop over \n */
+				p++;
+				len--;
+			}
+			continue;
+		}
+
+		/* skip invalid lines */
+		if (c == NULL || q - c < 3 || c[1] != ' ')
+			continue;
+
+		/* NULL-terminate p and c, file should end with \n */
+		*q = '\0';
+		*c = '\0';
+		c += 2;         /* hop over ": " */
+		if (len > 0) {  /* hop over \n */
+			q++;
+			len--;
+		}
+
+		if (strcmp(p, "REPO") == 0) { /* from global section in older files */
+			ctx->repo = c;
+		} else if (strcmp(p, "CPV") == 0) {
+			if (atom != NULL)
+				atom_implode(atom);
+			atom = atom_explode(c);
+#define match_key(X) match_key2(X,X)
+#define match_key2(X,Y) \
+		} else if (strcmp(p, #X) == 0) { \
+			meta->Y = c
+		match_key(DEFINED_PHASES);
+		match_key(DEPEND);
+		match_key2(DESC, DESCRIPTION);
+		match_key(EAPI);
+		match_key(IUSE);
+		match_key(KEYWORDS);
+		match_key(LICENSE);
+		match_key2(MD5, _md5_);
+		match_key2(SHA1, _eclasses_);
+		match_key(RDEPEND);
+		match_key(SLOT);
+		match_key(USE);
+		match_key(PDEPEND);
+#undef match_key
+#undef match_key2
+		}
+
+		p = q;
+	} while (len > 0);
+
+	/* ensure we don't free a garbage pointer */
+	ctx->repo = NULL;
+
+	return ret;
+}
+
 int
 tree_foreach_pkg(tree_ctx *ctx,
 		tree_pkg_cb callback, void *priv, tree_cat_filter filter,
@@ -964,6 +1174,10 @@ tree_foreach_pkg(tree_ctx *ctx,
 	if (ctx == NULL)
 		return EXIT_FAILURE;
 
+	/* handle Packages (binpkgs index) file separately */
+	if (ctx->cachetype == CACHE_PACKAGES)
+		return tree_foreach_packages(ctx, callback, priv);
+
 	ctx->do_sort = sort;
 	if (catsortfunc != NULL)
 		ctx->catsortfunc = catsortfunc;
@@ -1009,23 +1223,35 @@ tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete)
 							&pkg_ctx->repo, &pkg_ctx->repo_len);
 				pkg_ctx->atom->REPO = pkg_ctx->repo;
 			}
-		} else { /* metadata or ebuild */
+		} else { /* metadata, ebuild, binpkg or Packages */
+			tree_pkg_meta *meta = NULL;
 			if (pkg_ctx->atom->SLOT == NULL) {
 				if (pkg_ctx->slot == NULL) {
-					tree_pkg_meta *meta = tree_pkg_read(pkg_ctx);
+					meta = tree_pkg_read(pkg_ctx);
 					if (meta != NULL) {
 						if (meta->SLOT != NULL) {
 							pkg_ctx->slot = xstrdup(meta->SLOT);
 							pkg_ctx->slot_len = strlen(pkg_ctx->slot);
 						}
-						tree_close_meta(meta);
 					}
 				}
 				pkg_ctx->atom->SLOT = pkg_ctx->slot;
 			}
 			/* repo is set from the tree, when found */
-			if (pkg_ctx->atom->REPO == NULL)
+			if (pkg_ctx->atom->REPO == NULL) {
+				if (pkg_ctx->repo == NULL && ctx->cachetype == CACHE_BINPKGS) {
+					if (meta == NULL)
+						meta = tree_pkg_read(pkg_ctx);
+					if (meta != NULL && meta->repository != NULL) {
+						pkg_ctx->repo = xstrdup(meta->repository);
+						pkg_ctx->repo_len = strlen(pkg_ctx->repo);
+					}
+				}
 				pkg_ctx->atom->REPO = pkg_ctx->repo;
+			}
+
+			if (meta != NULL)
+				tree_close_meta(meta);
 		}
 
 		/* this is a bit atom territory, but since we pulled in SLOT we

diff --git a/libq/tree.h b/libq/tree.h
index c2a30f1..d769b7b 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -18,7 +18,7 @@ typedef struct tree_pkg_ctx      tree_pkg_ctx;
 typedef struct tree_pkg_meta     tree_pkg_meta;
 typedef struct tree_metadata_xml tree_metadata_xml;
 
-/* VDB context */
+/* tree context */
 struct tree_ctx {
 	int portroot_fd;
 	int tree_fd;
@@ -35,11 +35,15 @@ struct tree_ctx {
 		CACHE_METADATA_PMS,
 		CACHE_EBUILD,
 		CACHE_VDB,
+		CACHE_PACKAGES,
+		CACHE_BINPKGS,
 	} cachetype:3;
 	tree_pkg_ctx *ebuilddir_pkg_ctx;
 	tree_cat_ctx *ebuilddir_cat_ctx;
 	tree_ctx *ebuilddir_ctx;
 	char *repo;
+	char *pkgs;
+	size_t pkgslen;
 };
 
 /* Category context */
@@ -90,6 +94,10 @@ struct tree_pkg_meta {
 	char *BDEPEND;
 	char *_eclasses_;
 	char *_md5_;
+	/* binpkgs/vdb */
+	char *CONTENTS;
+	char *USE;
+	char *repository;
 };
 
 /* Metadata.xml */
@@ -104,8 +112,9 @@ struct tree_metadata_xml {
 typedef int (tree_pkg_cb)(tree_pkg_ctx *, void *priv);
 typedef int (tree_cat_filter)(tree_cat_ctx *, void *priv);
 
-tree_ctx *tree_open_vdb(const char *sroot, const char *svdb);
 tree_ctx *tree_open(const char *sroot, const char *portdir);
+tree_ctx *tree_open_vdb(const char *sroot, const char *svdb);
+tree_ctx *tree_open_binpkg(const char *sroot, const char *spkg);
 void tree_close(tree_ctx *ctx);
 int tree_filter_cat(const struct dirent *de);
 tree_cat_ctx *tree_open_cat(tree_ctx *ctx, const char *name);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     58dee81a9d47a97fc3d4175ed491dd6bc942a08c
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Nov 13 18:09:50 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Nov 13 18:09:50 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=58dee81a

libq/set: add cnt_set to return number of elements

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/set.c | 6 ++++++
 libq/set.h | 1 +
 2 files changed, 7 insertions(+)

diff --git a/libq/set.c b/libq/set.c
index a934a0d..e0ea396 100644
--- a/libq/set.c
+++ b/libq/set.c
@@ -268,6 +268,12 @@ values_set(set *q, array_t *ret)
 	return q->len;
 }
 
+size_t
+cnt_set(set *q)
+{
+	return q == NULL ? 0 : q->len;
+}
+
 /* clear out a set */
 void
 clear_set(set *q)

diff --git a/libq/set.h b/libq/set.h
index 00ef909..638cc15 100644
--- a/libq/set.h
+++ b/libq/set.h
@@ -37,6 +37,7 @@ void *get_set(const char *name, set *q);
 set *del_set(const char *s, set *q, bool *removed);
 size_t list_set(set *q, char ***l);
 size_t values_set(set *q, array_t *ret);
+size_t cnt_set(set *q);
 void free_set(set *q);
 void clear_set(set *q);
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     bb7ad62a52e63cc3c02b799d78cd47a6c8cf1c35
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Nov 13 15:47:40 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Nov 13 15:47:40 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=bb7ad62a

libq/xpak: fix copy/paste typo causing xpak_extract to always fail

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xpak.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libq/xpak.c b/libq/xpak.c
index ee989e4..c2e93f9 100644
--- a/libq/xpak.c
+++ b/libq/xpak.c
@@ -192,8 +192,8 @@ xpak_extract(
 	/* the xpak may be large (like when it has CONTENTS) #300744 */
 	x->data = (size_t)x->data_len < sizeof(ext) ? ext : xmalloc(x->data_len);
 	in = fread(x->data, 1, x->data_len, x->fp);
-	if (in != (size_t)x->index_len)
-		err("insufficient data read, got %zd, requested %d", in, x->index_len);
+	if (in != (size_t)x->data_len)
+		err("insufficient data read, got %zd, requested %d", in, x->data_len);
 
 	_xpak_walk_index(x, argc, argv, func);
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     58aaf6e646f22e2f05599f54b13400812afa5a79
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Nov 13 15:19:23 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Nov 13 15:19:23 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=58aaf6e6

libq/xpak: turn asserts into real error checks

Using asserts to validate external data is a bad idea.  Turn them into
proper errors instead.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xpak.c | 23 +++++++++++++----------
 1 file changed, 13 insertions(+), 10 deletions(-)

diff --git a/libq/xpak.c b/libq/xpak.c
index 2785899..ee989e4 100644
--- a/libq/xpak.c
+++ b/libq/xpak.c
@@ -11,7 +11,6 @@
 #include <stdio.h>
 #include <string.h>
 #include <xalloc.h>
-#include <assert.h>
 
 #include "basename.h"
 #include "copy_file.h"
@@ -61,7 +60,9 @@ static void _xpak_walk_index(
 	p = x->index;
 	while ((p - x->index) < x->index_len) {
 		pathname_len = READ_BE_INT32((unsigned char*)p);
-		assert((size_t)pathname_len < sizeof(pathname));
+		if (pathname_len >= sizeof(pathname))
+			err("pathname length %d exceeds limit %zd",
+					pathname_len, sizeof(pathname));
 		p += 4;
 		memcpy(pathname, p, pathname_len);
 		pathname[pathname_len] = '\0';
@@ -151,9 +152,11 @@ xpak_list(
 
 	x->dir_fd = dir_fd;
 	x->index = buf;
-	assert((size_t)x->index_len < sizeof(buf));
+	if (x->index_len >= sizeof(buf))
+		err("index length %d exceeds limit %zd", x->index_len, sizeof(buf));
 	ret = fread(x->index, 1, x->index_len, x->fp);
-	assert(ret == (size_t)x->index_len);
+	if (ret != (size_t)x->index_len)
+		err("insufficient data read, got %zd, requested %d", ret, x->index_len);
 	_xpak_walk_index(x, argc, argv, func);
 
 	_xpak_close(x);
@@ -180,17 +183,17 @@ xpak_extract(
 	x->dir_fd = dir_fd;
 	x->index = buf;
 
-	assert((size_t)x->index_len < sizeof(buf));
+	if (x->index_len >= sizeof(buf))
+		err("index length %d exceeds limit %zd", x->index_len, sizeof(buf));
 	in = fread(x->index, 1, x->index_len, x->fp);
-	if ((int)in != x->index_len)
-		err("index chunk: read %i bytes, wanted %i bytes",
-				(int)in, x->index_len);
+	if (in != (size_t)x->index_len)
+		err("insufficient data read, got %zd, requested %d", in, x->index_len);
 
 	/* the xpak may be large (like when it has CONTENTS) #300744 */
 	x->data = (size_t)x->data_len < sizeof(ext) ? ext : xmalloc(x->data_len);
 	in = fread(x->data, 1, x->data_len, x->fp);
-	if ((int)in != x->data_len)
-		err("data chunk: read %i bytes, wanted %i bytes", (int)in, x->data_len);
+	if (in != (size_t)x->index_len)
+		err("insufficient data read, got %zd, requested %d", in, x->index_len);
 
 	_xpak_walk_index(x, argc, argv, func);
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     ff773ed5a95fe7d6891455dbee7cd686c4ceef97
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Nov  9 10:27:41 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Nov  9 10:27:41 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ff773ed5

libq/tree: fix crash in tree_get_atom for ebuild mode with SLOT

When an ebuild doesn't contain SLOT, but it was requested, just set SLOT
to NULL, instead of trying to strdup(NULL).

Thanks Joakim Tjernlund

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index a8baabe..b0d0b5b 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1014,8 +1014,10 @@ tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete)
 				if (pkg_ctx->slot == NULL) {
 					tree_pkg_meta *meta = tree_pkg_read(pkg_ctx);
 					if (meta != NULL) {
-						pkg_ctx->slot = xstrdup(meta->SLOT);
-						pkg_ctx->slot_len = strlen(pkg_ctx->slot);
+						if (meta->SLOT != NULL) {
+							pkg_ctx->slot = xstrdup(meta->SLOT);
+							pkg_ctx->slot_len = strlen(pkg_ctx->slot);
+						}
 						tree_close_meta(meta);
 					}
 				}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     debecebe010e3c33e594ec5745e591982095d62b
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Sep 26 13:53:18 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Sep 26 13:53:18 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=debecebe

libq/atom: add atom_compar_cb, qsort compatible comparator function

use this new function from libq/tree where it was moved from

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 17 +++++++++++++++++
 libq/atom.h |  1 +
 libq/tree.c |  9 +--------
 3 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 8f564eb..f80bb7d 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -843,3 +843,20 @@ atom_format(const char *format, const depend_atom *atom)
 {
 	return atom_format_r(_atom_buf, sizeof(_atom_buf), format, atom);
 }
+
+/* qsort compatible callback function */
+inline int
+atom_compar_cb(const void *l, const void *r)
+{
+	const depend_atom *al = l;
+	const depend_atom *ar = r;
+
+	switch (atom_compare(al, ar)) {
+		case EQUAL:  return  0;
+		case NEWER:  return -1;
+		case OLDER:  return  1;
+		default:     return strcmp(al->PN, ar->PN);
+	}
+
+	/* unreachable */
+}

diff --git a/libq/atom.h b/libq/atom.h
index 43397ad..a5175b0 100644
--- a/libq/atom.h
+++ b/libq/atom.h
@@ -104,5 +104,6 @@ char *atom_format_r(char *buf, size_t buflen,
 		const char *format, const depend_atom *atom);
 char *atom_to_string(depend_atom *a);
 char *atom_format(const char *format, const depend_atom *atom);
+int atom_compar_cb(const void *l, const void *r);
 
 #endif

diff --git a/libq/tree.c b/libq/tree.c
index 8caed00..a8baabe 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -320,14 +320,7 @@ tree_pkg_compar(const void *l, const void *r)
 	depend_atom *al = tree_get_atom(pl, false);
 	depend_atom *ar = tree_get_atom(pr, false);
 
-	switch (atom_compare(al, ar)) {
-		case EQUAL:  return  0;
-		case NEWER:  return -1;
-		case OLDER:  return  1;
-		default:     return strcmp(al->PN, ar->PN);
-	}
-
-	/* unreachable */
+	return atom_compar_cb(al, ar);
 }
 
 static tree_pkg_ctx *

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     d500e279f02bd5b58407975da145b1334c33e696
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Sep 26 13:22:24 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Sep 26 13:22:24 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=d500e279

libq/tree: simplify atom construction in tree_get_atoms_cb

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index f8d90ce..8caed00 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1061,10 +1061,9 @@ static int tree_get_atoms_cb(tree_pkg_ctx *pkg_ctx, void *priv)
 	char abuf[BUFSIZ];
 
 	if (state->fullcpv) {
-		size_t len = snprintf(abuf, sizeof(abuf), "%s/%s-%s",
-				atom->CATEGORY, atom->PN, atom->PV);
-		if (atom->PR_int > 0)
-			snprintf(abuf + len, sizeof(abuf) - len, "-r%d", atom->PR_int);
+		snprintf(abuf, sizeof(abuf), "%s/%s-%s",
+				atom->CATEGORY, atom->PN,
+				atom->PR_int > 0 ? atom->PVR : atom->PV);
 		state->cpf = add_set(abuf, state->cpf);
 	} else {
 		snprintf(abuf, sizeof(abuf), "%s/%s", atom->CATEGORY, atom->PN);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     21a3d43cf7f43040132e9035c7b095a17b5a8bdd
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Sep 26 13:52:33 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Sep 26 13:52:33 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=21a3d43c

libq/xarray: add xarraysort function

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xarray.c | 5 +++++
 libq/xarray.h | 1 +
 2 files changed, 6 insertions(+)

diff --git a/libq/xarray.c b/libq/xarray.c
index a5cac49..49b478b 100644
--- a/libq/xarray.c
+++ b/libq/xarray.c
@@ -44,6 +44,11 @@ void *xarraypush(array_t *arr, const void *ele, size_t ele_len)
 	return xarraypush_ptr(arr, xmemdup(ele, ele_len));
 }
 
+void xarraysort(array_t *arr, int (*compar)(const void *, const void *))
+{
+	qsort(arr->eles, arr->num, sizeof(void *), compar);
+}
+
 void xarraydelete_ptr(array_t *arr, size_t elem)
 {
 	arr->num--;

diff --git a/libq/xarray.h b/libq/xarray.h
index 71dfecb..6af0d39 100644
--- a/libq/xarray.h
+++ b/libq/xarray.h
@@ -40,6 +40,7 @@ typedef struct {
 void *xarrayget(array_t *arr, size_t idx);
 void *xarraypush_ptr(array_t *arr, void *ele);
 void *xarraypush(array_t *arr, const void *ele, size_t ele_len);
+void xarraysort(array_t *arr, int (*compar)(const void *, const void *));
 void xarraydelete_ptr(array_t *arr, size_t elem);
 void xarraydelete(array_t *arr, size_t elem);
 void xarrayfree_int(array_t *arr);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     f60d48f824936a052167225bc41f01a8078bce5e
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Sep 26 14:05:06 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Sep 26 14:05:06 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=f60d48f8

libq/atom: refine atom_compar_cb behaviour

- also consider CATEGORY while sorting (humans expect this)
- sort PN caseINsensitive (for Python and Perl pkgs which aren't
  consistent)

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/libq/atom.c b/libq/atom.c
index f80bb7d..6e12c0a 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -855,7 +855,14 @@ atom_compar_cb(const void *l, const void *r)
 		case EQUAL:  return  0;
 		case NEWER:  return -1;
 		case OLDER:  return  1;
-		default:     return strcmp(al->PN, ar->PN);
+		default:
+		{
+			int ret;
+			ret = strcmp(al->CATEGORY, ar->CATEGORY);
+			if (ret == 0)
+				ret = strcasecmp(al->PN, ar->PN);
+			return ret;
+		}
 	}
 
 	/* unreachable */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     84c003caddc16e7455d6d8d07d60d1c868ea5aa1
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Sep 26 12:59:46 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Sep 26 12:59:46 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=84c003ca

libq/tree: fix tree_get_atoms not to use atom_format

atom_format uses colour escapes, which messes up the hashing used in the
set.

Thanks Alexander Wetzel.

Bug: https://bugs.gentoo.org/695586
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 2797450..f8d90ce 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1058,12 +1058,17 @@ static int tree_get_atoms_cb(tree_pkg_ctx *pkg_ctx, void *priv)
 {
 	struct get_atoms_state *state = (struct get_atoms_state *)priv;
 	depend_atom *atom = tree_get_atom(pkg_ctx, false);
+	char abuf[BUFSIZ];
 
 	if (state->fullcpv) {
-		state->cpf = add_set(atom_format("%[CATEGORY]%[PF]", atom), state->cpf);
+		size_t len = snprintf(abuf, sizeof(abuf), "%s/%s-%s",
+				atom->CATEGORY, atom->PN, atom->PV);
+		if (atom->PR_int > 0)
+			snprintf(abuf + len, sizeof(abuf) - len, "-r%d", atom->PR_int);
+		state->cpf = add_set(abuf, state->cpf);
 	} else {
-		state->cpf = add_set_unique(atom_format("%[CATEGORY]%[PN]", atom),
-				state->cpf, NULL);
+		snprintf(abuf, sizeof(abuf), "%s/%s", atom->CATEGORY, atom->PN);
+		state->cpf = add_set_unique(abuf, state->cpf, NULL);
 	}
 
 	return 0;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     5826ee815228775b61ed1587e1346f764dc93945
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Sep 25 15:04:23 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Sep 25 15:04:23 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=5826ee81

libq/tree: don't close tree in tree_get_atoms

The caller passes tree, so the caller should clean it up.  This fixes
double frees observed in bug #695586

Bug: https://bugs.gentoo.org/695586
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index 0699334..2797450 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1078,7 +1078,6 @@ tree_get_atoms(tree_ctx *ctx, bool fullcpv, set *satoms)
 	};
 
 	tree_foreach_pkg_fast(ctx, tree_get_atoms_cb, &state, NULL);
-	tree_close(ctx);
 
 	return state.cpf;
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     cf086cd6b3c452729f68fd9ce3411e28976ae94e
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Sep 21 19:48:29 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Sep 21 19:48:29 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=cf086cd6

libq/set: add funcs for key/value support

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/set.c    | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++--------
 libq/set.h    |  10 ++++--
 libq/xarray.h |   4 +--
 3 files changed, 100 insertions(+), 17 deletions(-)

diff --git a/libq/set.c b/libq/set.c
index f2c9394..a934a0d 100644
--- a/libq/set.c
+++ b/libq/set.c
@@ -18,7 +18,7 @@
 #include "set.h"
 
 static unsigned int
-fnv1a32(char *s)
+fnv1a32(const char *s)
 {
 	unsigned int ret = 2166136261UL;
 	for (; *s != '\0'; s++)
@@ -33,7 +33,7 @@ create_set(void)
 	return xzalloc(sizeof(set));
 }
 
-/* add elem to a set (unpure: could add duplicates) */
+/* add elem to a set (unpure: could add duplicates, basically hash) */
 set *
 add_set(const char *name, set *q)
 {
@@ -47,6 +47,7 @@ add_set(const char *name, set *q)
 	ll->next = NULL;
 	ll->name = xstrdup(name);
 	ll->hash = fnv1a32(ll->name);
+	ll->val = NULL;
 
 	pos = ll->hash % _SET_HASH_SIZE;
 	if (q->buckets[pos] == NULL) {
@@ -61,11 +62,10 @@ add_set(const char *name, set *q)
 	return q;
 }
 
-/* add elem to set if it doesn't exist yet (pure definition of set) */
+/* add elem to set if it doesn't exist yet (pure definition of hash) */
 set *
 add_set_unique(const char *name, set *q, bool *unique)
 {
-	char *mname = xstrdup(name);
 	unsigned int hash;
 	int pos;
 	elem *ll;
@@ -75,20 +75,20 @@ add_set_unique(const char *name, set *q, bool *unique)
 	if (q == NULL)
 		q = create_set();
 
-	hash = fnv1a32(mname);
+	hash = fnv1a32(name);
 	pos = hash % _SET_HASH_SIZE;
 
 	if (q->buckets[pos] == NULL) {
 		q->buckets[pos] = ll = xmalloc(sizeof(*ll));
 		ll->next = NULL;
-		ll->name = mname;
+		ll->name = xstrdup(name);
 		ll->hash = hash;
+		ll->val = NULL;
 		uniq = true;
 	} else {
 		ll = NULL;
 		for (w = q->buckets[pos]; w != NULL; ll = w, w = w->next) {
-			if (w->hash == hash && strcmp(w->name, mname) == 0) {
-				free(mname);
+			if (w->hash == hash && strcmp(w->name, name) == 0) {
 				uniq = false;
 				break;
 			}
@@ -96,8 +96,9 @@ add_set_unique(const char *name, set *q, bool *unique)
 		if (w == NULL) {
 			ll = ll->next = xmalloc(sizeof(*ll));
 			ll->next = NULL;
-			ll->name = mname;
+			ll->name = xstrdup(name);
 			ll->hash = hash;
+			ll->val = NULL;
 			uniq = true;
 		}
 	}
@@ -109,22 +110,60 @@ add_set_unique(const char *name, set *q, bool *unique)
 	return q;
 }
 
+/* add ptr to set with name as key, return existing value when key
+ * already exists or NULL otherwise */
+void *
+add_set_value(const char *name, void *ptr, set *q)
+{
+	unsigned int hash;
+	int pos;
+	elem *ll;
+	elem *w;
+
+	hash = fnv1a32(name);
+	pos = hash % _SET_HASH_SIZE;
+
+	if (q->buckets[pos] == NULL) {
+		q->buckets[pos] = ll = xmalloc(sizeof(*ll));
+		ll->next = NULL;
+		ll->name = xstrdup(name);
+		ll->hash = hash;
+		ll->val = ptr;
+	} else {
+		ll = NULL;
+		for (w = q->buckets[pos]; w != NULL; ll = w, w = w->next) {
+			if (w->hash == hash && strcmp(w->name, name) == 0)
+				return w->val;
+		}
+		if (w == NULL) {
+			ll = ll->next = xmalloc(sizeof(*ll));
+			ll->next = NULL;
+			ll->name = xstrdup(name);
+			ll->hash = hash;
+			ll->val = ptr;
+		}
+	}
+
+	q->len++;
+	return NULL;
+}
+
 /* returns whether s is in set */
 bool
-contains_set(char *s, set *q)
+contains_set(const char *name, set *q)
 {
 	unsigned int hash;
 	int pos;
 	elem *w;
 	bool found;
 
-	hash = fnv1a32(s);
+	hash = fnv1a32(name);
 	pos = hash % _SET_HASH_SIZE;
 
 	found = false;
 	if (q->buckets[pos] != NULL) {
 		for (w = q->buckets[pos]; w != NULL; w = w->next) {
-			if (w->hash == hash && strcmp(w->name, s) == 0) {
+			if (w->hash == hash && strcmp(w->name, name) == 0) {
 				found = true;
 				break;
 			}
@@ -134,9 +173,31 @@ contains_set(char *s, set *q)
 	return found;
 }
 
+/* returns the value for name, or NULL if not found (cannot
+ * differentiate between value NULL and unset) */
+void *
+get_set(const char *name, set *q)
+{
+	unsigned int hash;
+	int pos;
+	elem *w;
+
+	hash = fnv1a32(name);
+	pos = hash % _SET_HASH_SIZE;
+
+	if (q->buckets[pos] != NULL) {
+		for (w = q->buckets[pos]; w != NULL; w = w->next) {
+			if (w->hash == hash && strcmp(w->name, name) == 0)
+				return w->val;
+		}
+	}
+
+	return NULL;
+}
+
 /* remove elem from a set. matches ->name and frees name,item */
 set *
-del_set(char *s, set *q, bool *removed)
+del_set(const char *s, set *q, bool *removed)
 {
 	unsigned int hash;
 	int pos;
@@ -191,6 +252,22 @@ list_set(set *q, char ***l)
 	return q->len;
 }
 
+size_t
+values_set(set *q, array_t *ret)
+{
+	int i;
+	elem *w;
+	array_t blank = array_init_decl;
+
+	*ret = blank;
+	for (i = 0; i < _SET_HASH_SIZE; i++) {
+		for (w = q->buckets[i]; w != NULL; w = w->next)
+			xarraypush_ptr(ret, w->val);
+	}
+
+	return q->len;
+}
+
 /* clear out a set */
 void
 clear_set(set *q)

diff --git a/libq/set.h b/libq/set.h
index 7ca5f65..00ef909 100644
--- a/libq/set.h
+++ b/libq/set.h
@@ -10,12 +10,15 @@
 #include <stdbool.h>
 #include <unistd.h>
 
+#include "xarray.h"
+
 typedef struct elem_t elem;
 typedef struct set_t set;
 
 struct elem_t {
 	char *name;
 	unsigned int hash;  /* FNV1a32 */
+	void *val;
 	elem *next;
 };
 
@@ -28,9 +31,12 @@ struct set_t {
 set *create_set(void);
 set *add_set(const char *name, set *q);
 set *add_set_unique(const char *name, set *q, bool *unique);
-bool contains_set(char *s, set *q);
-set *del_set(char *s, set *q, bool *removed);
+void *add_set_value(const char *name, void *ptr, set *q);
+bool contains_set(const char *name, set *q);
+void *get_set(const char *name, set *q);
+set *del_set(const char *s, set *q, bool *removed);
 size_t list_set(set *q, char ***l);
+size_t values_set(set *q, array_t *ret);
 void free_set(set *q);
 void clear_set(set *q);
 

diff --git a/libq/xarray.h b/libq/xarray.h
index 22ee47a..71dfecb 100644
--- a/libq/xarray.h
+++ b/libq/xarray.h
@@ -26,9 +26,9 @@ typedef struct {
  */
 /* TODO: remove ele = NULL after checking all consumers don't rely on this */
 #define array_for_each(arr, n, ele) \
-	for (n = 0, ele = NULL; n < array_cnt(arr) && (ele = arr->eles[n]); n++)
+	for (n = 0, ele = NULL; n < array_cnt(arr) && (ele = (arr)->eles[n]); n++)
 #define array_for_each_rev(arr, n, ele) \
-	for (n = array_cnt(arr); n-- > 0 && (ele = arr->eles[n]); /*nothing*/)
+	for (n = array_cnt(arr); n-- > 0 && (ele = (arr)->eles[n]); /*nothing*/)
 #define array_get_elem(arr, n) (arr->eles[n])
 #define array_init_decl { .eles = NULL, .num = 0, }
 #define array_cnt(arr) (arr)->num

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     ee31ca7a1155fa1c9ffd605929985e77b66f5cb8
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Sep 20 17:10:52 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Sep 20 17:10:52 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ee31ca7a

libq/atom: return compare results as enum proper

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c |  8 ++++----
 libq/atom.h | 12 +++++++++---
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 4c21a40..8f564eb 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -339,7 +339,7 @@ atom_implode(depend_atom *atom)
 	free(atom);
 }
 
-static int
+static atom_equality
 _atom_compare_match(int ret, atom_operator op)
 {
 	if (op == ATOM_OP_NONE)
@@ -370,7 +370,7 @@ _atom_compare_match(int ret, atom_operator op)
  * foo-1 <OLDER> foo-2
  * foo-1 <NOT_EQUAL> bar-1
  */
-int
+atom_equality
 atom_compare(const depend_atom *data, const depend_atom *query)
 {
 	atom_operator pfx_op;
@@ -620,11 +620,11 @@ atom_compare(const depend_atom *data, const depend_atom *query)
 		return _atom_compare_match(NEWER, pfx_op);
 }
 
-int
+atom_equality
 atom_compare_str(const char * const s1, const char * const s2)
 {
 	depend_atom *a1, *a2;
-	int ret = ERROR;
+	atom_equality ret = ERROR;
 
 	a1 = atom_explode(s1);
 	if (!a1)

diff --git a/libq/atom.h b/libq/atom.h
index 72266d5..43397ad 100644
--- a/libq/atom.h
+++ b/libq/atom.h
@@ -87,12 +87,18 @@ typedef struct {
 } depend_atom;
 
 extern const char * const booga[];
-enum { ERROR=0, NOT_EQUAL, EQUAL, NEWER, OLDER };
+typedef enum {
+	ERROR = 0,
+	NOT_EQUAL,
+	EQUAL,
+	NEWER,
+	OLDER
+} atom_equality;
 
 depend_atom *atom_explode(const char *atom);
 void atom_implode(depend_atom *atom);
-int atom_compare(const depend_atom *a1, const depend_atom *a2);
-int atom_compare_str(const char * const s1, const char * const s2);
+atom_equality atom_compare(const depend_atom *a1, const depend_atom *a2);
+atom_equality atom_compare_str(const char * const s1, const char * const s2);
 char *atom_to_string_r(char *buf, size_t buflen, depend_atom *a);
 char *atom_format_r(char *buf, size_t buflen,
 		const char *format, const depend_atom *atom);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     a496627e69834aae3ec560d500347b26769346b7
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Jul 14 18:50:50 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Jul 14 18:50:50 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=a496627e

libq/atom: make atom_format return <unset> for SLOT and SUBSLOT

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 170eeea..4c21a40 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -778,14 +778,14 @@ atom_format_r(
 						append_buf(buf, buflen, "%s%s%s%s",
 								YELLOW,
 								connected ? ":" : "",
-								atom->SLOT ? atom->SLOT : "<unset>",
+								HN(atom->SLOT),
 								NORM);
 				} else if (!strncmp("SUBSLOT", fmt, len)) {
 					if (showit || atom->SUBSLOT)
 						append_buf(buf, buflen, "%s%s%s%s%s",
 								YELLOW,
 								connected ? "/" : "",
-								atom->SUBSLOT ? atom->SUBSLOT : "",
+								HN(atom->SUBSLOT),
 								atom_slotdep_str[atom->slotdep],
 								NORM);
 				} else if (!strncmp("REPO", fmt, len)) {

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     61c865750c821381f2f8d3bc93b4e149127d2fdb
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Jul 13 15:32:46 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Jul 13 15:32:46 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=61c86575

libq/tree: ensure we don't work on garbage on sorted pkg trees

The contents of dirents come from a static buffer that may get
re-purposed when the next readdir call is made, so we cannot rely on it
staying around.  In particular on large directories, the entries will
get recycled, and hence garbage appear.  Thus, we need to copy the
entries, and free those copies.  The behaviour before
f855d0f4f7c3e6e570a1ad3dc98d737e78996e4a was actually using scandir
which allocates space for all dirents.  We now basically just copy the
bit we need, instead of the full dirent.

Thanks Sergei Trofimovich (slyfox) for digging into this case and
providing a VDB which displayed the problem.

Bug: https://bugs.gentoo.org/689290
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 99e864b..0699334 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -344,6 +344,8 @@ tree_next_pkg_int(tree_cat_ctx *cat_ctx)
 			cat_ctx->pkg_cnt = 0;
 			cat_ctx->pkg_cur = 0;
 			while ((de = readdir(cat_ctx->dir)) != NULL) {
+				char *name;
+
 				if (tree_filter_pkg(de) == 0)
 					continue;
 
@@ -352,10 +354,13 @@ tree_next_pkg_int(tree_cat_ctx *cat_ctx)
 					cat_ctx->pkg_ctxs = xrealloc(cat_ctx->pkg_ctxs,
 								sizeof(*cat_ctx->pkg_ctxs) * pkg_size);
 				}
+				name = xstrdup(de->d_name);
 				pkg_ctx = cat_ctx->pkg_ctxs[cat_ctx->pkg_cnt++] =
-					tree_open_pkg(cat_ctx, de->d_name);
-				if (pkg_ctx == NULL)
+					tree_open_pkg(cat_ctx, name);
+				if (pkg_ctx == NULL) {
+					free(name);
 					cat_ctx->pkg_cnt--;
+				}
 			}
 
 			if (cat_ctx->ctx->pkgsortfunc != NULL && cat_ctx->pkg_cnt > 1) {
@@ -948,6 +953,8 @@ tree_close_pkg(tree_pkg_ctx *pkg_ctx)
 	/* avoid freeing tree_ctx' repo */
 	if (pkg_ctx->cat_ctx->ctx->repo != pkg_ctx->repo)
 		free(pkg_ctx->repo);
+	if (pkg_ctx->cat_ctx->ctx->do_sort)
+		free((char *)pkg_ctx->name);
 	free(pkg_ctx->slot);
 	free(pkg_ctx);
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     1b4c0049ff8a9297d0242c87d74946508a58c75c
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Jul 13 09:39:11 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Jul 13 09:39:11 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=1b4c0049

libq/tree: greatly simplify tree_get_vdb_atoms

use tree_foreach_pkg_fast() to construct the set of string atom names
using atom_format()

this function likely should be removed as the consumer should work on
the atoms instead of their string representation

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 91 ++++++++++++++++++-------------------------------------------
 1 file changed, 27 insertions(+), 64 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 6bb6a89..f4314d0 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1042,77 +1042,40 @@ tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete)
 	return pkg_ctx->atom;
 }
 
-set *
-tree_get_vdb_atoms(const char *sroot, const char *svdb, int fullcpv)
-{
-	tree_ctx *ctx;
+struct get_vdb_atoms_state {
+	set *cpf;
+	bool fullcpv;
+};
 
-	int cfd, j;
-	int dfd, i;
+static int tree_get_vdb_atoms_cb(tree_pkg_ctx *pkg_ctx, void *priv)
+{
+	struct get_vdb_atoms_state *state = (struct get_vdb_atoms_state *)priv;
+	depend_atom *atom = tree_get_atom(pkg_ctx, false);
 
-	char buf[_Q_PATH_MAX];
-	char slot[_Q_PATH_MAX];
-	char *slotp = slot;
-	size_t slot_len;
+	if (state->fullcpv) {
+		state->cpf = add_set(atom_format("%[CATEGORY]%[PF]", atom), state->cpf);
+	} else {
+		state->cpf = add_set_unique(atom_format("%[CATEGORY]%[PN]", atom),
+				state->cpf, NULL);
+	}
 
-	struct dirent **cat;
-	struct dirent **pf;
+	return 0;
+}
 
-	depend_atom *atom = NULL;
-	set *cpf = NULL;
+set *
+tree_get_vdb_atoms(const char *sroot, const char *svdb, int fullcpv)
+{
+	tree_ctx *ctx;
+	struct get_vdb_atoms_state state = {
+		.cpf = NULL,
+		.fullcpv = fullcpv != 0
+	};
 
 	ctx = tree_open_vdb(sroot, svdb);
 	if (!ctx)
 		return NULL;
-
-	/* scan the cat first */
-	cfd = scandirat(ctx->tree_fd, ".", &cat, tree_filter_cat, alphasort);
-	if (cfd < 0)
-		goto fuckit;
-
-	for (j = 0; j < cfd; j++) {
-		dfd = scandirat(ctx->tree_fd, cat[j]->d_name,
-				&pf, tree_filter_pkg, alphasort);
-		if (dfd < 0)
-			continue;
-		for (i = 0; i < dfd; i++) {
-			int blen = snprintf(buf, sizeof(buf), "%s/%s/SLOT",
-					cat[j]->d_name, pf[i]->d_name);
-			if (blen < 0 || (size_t)blen >= sizeof(buf)) {
-				warnf("unable to parse long package: %s/%s",
-						cat[j]->d_name, pf[i]->d_name);
-				continue;
-			}
-
-			/* Chop the SLOT for the atom parsing. */
-			buf[blen - 5] = '\0';
-			if ((atom = atom_explode(buf)) == NULL)
-				continue;
-			/* Restore the SLOT. */
-			buf[blen - 5] = '/';
-
-			slot_len = sizeof(slot);
-			eat_file_at(ctx->tree_fd, buf, &slotp, &slot_len);
-			rmspace(slot);
-
-			if (fullcpv) {
-				if (atom->PR_int)
-					snprintf(buf, sizeof(buf), "%s/%s-%s-r%i",
-							atom->CATEGORY, atom->PN, atom->PV, atom->PR_int);
-				else
-					snprintf(buf, sizeof(buf), "%s/%s-%s",
-							atom->CATEGORY, atom->PN, atom->PV);
-			} else {
-				snprintf(buf, sizeof(buf), "%s/%s", atom->CATEGORY, atom->PN);
-			}
-			atom_implode(atom);
-			cpf = add_set(buf, cpf);
-		}
-		scandir_free(pf, dfd);
-	}
-	scandir_free(cat, cfd);
-
- fuckit:
+	tree_foreach_pkg_fast(ctx, tree_get_vdb_atoms_cb, &state, NULL);
 	tree_close(ctx);
-	return cpf;
+
+	return state.cpf;
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     c720262bfd9a31512b03f2e3129839886d9d27c6
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Jul 12 17:59:20 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Jul 12 17:59:20 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=c720262b

libq/tree: avoid calling qsort with empty set

Encountering an empty directory in tree_next_pkg_int will not populate
cat_ctx->pkg_ctxs, so avoid calling qsort with it.  While at it, avoid
calling it for a single entry too.

Bug: https://bugs.gentoo.org/689290#c7
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 9dfbe7d..6bb6a89 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -341,7 +341,6 @@ tree_next_pkg_int(tree_cat_ctx *cat_ctx)
 	if (cat_ctx->ctx->do_sort) {
 		if (cat_ctx->pkg_ctxs == NULL) {
 			size_t pkg_size = 0;
-			cat_ctx->pkg_ctxs = NULL;
 			cat_ctx->pkg_cnt = 0;
 			cat_ctx->pkg_cur = 0;
 			while ((de = readdir(cat_ctx->dir)) != NULL) {
@@ -359,7 +358,7 @@ tree_next_pkg_int(tree_cat_ctx *cat_ctx)
 					cat_ctx->pkg_cnt--;
 			}
 
-			if (cat_ctx->ctx->pkgsortfunc != NULL) {
+			if (cat_ctx->ctx->pkgsortfunc != NULL && cat_ctx->pkg_cnt > 1) {
 				qsort(cat_ctx->pkg_ctxs, cat_ctx->pkg_cnt,
 						sizeof(*cat_ctx->pkg_ctxs), cat_ctx->ctx->pkgsortfunc);
 			}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     035c62a44e976d328a31703b824a306d248ef197
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Jun 19 07:38:32 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Jun 19 07:38:32 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=035c62a4

libq/atom: document atom_to_string takes any matching prefix string

Basically any matching prefix string will work, so CATEGORY could be
CATE or C or anything inbetween the full name and its first character.
Care should be taken, S for instance, selects SLOT, not SUBSLOT, and it
won't warn either.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libq/atom.c b/libq/atom.c
index 8ce7f18..170eeea 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -687,6 +687,7 @@ atom_to_string_r(char *buf, size_t buflen, depend_atom *a)
  * The possible "keywords" are:
  *  CATEGORY  P  PN  PV  PVR  PF  PR  SLOT  SUBSLOT  REPO  USE
  *    - these are all the standard portage variables (so see ebuild(5))
+ *    - any prefix of these (e.g. CAT, CA, C) will match as well
  *  pfx - the version qualifier if set (e.g. > < = !)
  *  sfx - the version qualifier if set (e.g. *)
  */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     4307f26db5803a9f73d902bff47cafe25d3e7f1f
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Jun 10 08:42:26 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Jun 10 08:42:26 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=4307f26d

libq/tree: fix crash in tree_open on failure

just free repo, if set, the tree is obviously not there

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libq/tree.c b/libq/tree.c
index 17f51fc..9dfbe7d 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -114,7 +114,8 @@ tree_open(const char *sroot, const char *portdir)
 		return ret;
 	}
 
-	tree_close(ret);
+	if (repo != NULL)
+		free(repo);
 	warnf("could not open repository at %s (under root %s)", portdir, sroot);
 
 	return NULL;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e074e6fa71c94cea6a6693822f33fc3ea6afc1d1
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Jun  5 07:56:44 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Jun  5 07:56:44 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e074e6fa

libq/hash: fix compilation without HAVE_SSL and HAVE_BLAKE2B

Bug: https://bugs.gentoo.org/687374
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/hash.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/libq/hash.c b/libq/hash.c
index 8ce29d7..32d047a 100644
--- a/libq/hash.c
+++ b/libq/hash.c
@@ -131,6 +131,7 @@ hash_compute_file(
 
 	while ((len = fread(data, 1, sizeof(data), f)) > 0) {
 		*flen += len;
+#if defined(HAVE_SSL) || defined(HAVE_BLAKE2B)
 #pragma omp parallel sections
 		{
 #ifdef HAVE_SSL
@@ -158,9 +159,11 @@ hash_compute_file(
 			}
 #endif
 		}
+#endif /* HAVE_SSL || HAVE_BLAKE2B */
 	}
 	fclose(f);
 
+#if defined(HAVE_SSL) || defined(HAVE_BLAKE2B)
 #pragma omp parallel sections
 	{
 #ifdef HAVE_SSL
@@ -199,4 +202,5 @@ hash_compute_file(
 		}
 #endif
 	}
+#endif /* HAVE_SSL || HAVE_BLAKE2B */
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     2c10662ac7ec65cbeb2cec25ab492604202ea6b8
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue May 21 14:06:35 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue May 21 14:06:35 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=2c10662a

libq/hash: fix some warnings when libb2 or openssl isn't found

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/hash.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/libq/hash.c b/libq/hash.c
index 5b13d49..8ce29d7 100644
--- a/libq/hash.c
+++ b/libq/hash.c
@@ -106,9 +106,15 @@ hash_compute_file(
 	SHA256_CTX s256;
 	SHA512_CTX s512;
 	WHIRLPOOL_CTX whrl;
+#else
+	(void)sha256;
+	(void)sha512;
+	(void)whrlpl;
 #endif
 #ifdef HAVE_BLAKE2B
 	blake2b_state bl2b;
+#else
+	(void)blak2b;
 #endif
 
 	if ((f = fopen(fname, "r")) == NULL)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     06d24881ff95c0eca6945420675e0ab953fa43d3
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue May 14 20:15:13 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue May 14 20:15:13 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=06d24881

libq/tree: cleanup tree_pkg_vdb_eat a little

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 0659426..17f51fc 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -497,9 +497,14 @@ tree_pkg_vdb_eat(
 		char **bufptr,
 		size_t *buflen)
 {
-	int fd = tree_pkg_vdb_openat(pkg_ctx, file, O_RDONLY, 0);
-	bool ret = eat_file_fd(fd, bufptr, buflen);
-	rmspace(*bufptr);
+	int fd;
+	bool ret;
+
+	fd = tree_pkg_vdb_openat(pkg_ctx, file, O_RDONLY, 0);
+	ret = eat_file_fd(fd, bufptr, buflen);
+	if (ret)
+		rmspace(*bufptr);
+
 	if (fd != -1)
 		close(fd);
 	return ret;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     cf2affbb7bc8b84160f618b55cad0332c32b6d3a
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue May 14 20:14:31 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue May 14 20:14:31 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=cf2affbb

libq/dep: fix highlighting all deps when match atom array is empty

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/dep.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/libq/dep.c b/libq/dep.c
index 76d7b1e..49edf10 100644
--- a/libq/dep.c
+++ b/libq/dep.c
@@ -244,7 +244,9 @@ dep_print_tree(
 	if (root->type == DEP_OR)
 		fprintf(fp, "|| (");
 	if (root->info) {
-		if (hlatoms != NULL && root->type == DEP_NORM) {
+		if (hlatoms != NULL && array_cnt(hlatoms) > 0 &&
+				root->type == DEP_NORM)
+		{
 			size_t i;
 			depend_atom *m;
 			char *oslot;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     4a767991a765dcdacdd3c3239aec8c17356b9dd7
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat May 11 09:11:50 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat May 11 09:11:50 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=4a767991

libq/tree: make tree_get_atom split SLOT if appropriate

Now SLOT and SUBSLOT are true different things, ensure we parse it
properly.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/libq/tree.c b/libq/tree.c
index 8384458..0659426 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -1020,6 +1020,18 @@ tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete)
 			if (pkg_ctx->atom->REPO == NULL)
 				pkg_ctx->atom->REPO = pkg_ctx->repo;
 		}
+
+		/* this is a bit atom territory, but since we pulled in SLOT we
+		 * need to split it up in SLOT and SUBSLOT for atom_format to
+		 * behave properly, this may be redundant but this probably
+		 * isn't much of an issue */
+		if (pkg_ctx->atom->SUBSLOT == NULL && pkg_ctx->atom->SLOT != NULL) {
+			char *ptr;
+			if ((ptr = strchr(pkg_ctx->atom->SLOT, '/')) != NULL) {
+				*ptr++ = '\0';
+				pkg_ctx->atom->SUBSLOT = ptr;
+			}
+		}
 	}
 
 	return pkg_ctx->atom;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     0486e2a62cdce058a9a569940a93dae1f0442f1a
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat May 11 07:13:06 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat May 11 07:13:06 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=0486e2a6

libq/atom: split out SLOT and SUBSLOT for atom_format

it is a bit unclear whether SUBSLOT is part of SLOT or not, but PMS
doesn't allow '/' so probably it's supposed to be separate, hence split
them up, to format the former combo use %[SLOT]%[SUBSLOT]

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 0eaee5c..5627415 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -609,7 +609,7 @@ atom_to_string_r(char *buf, size_t buflen, depend_atom *a)
  *  %{keyword}: Always display the field that matches "keyword" or <unset>
  *  %[keyword]: Only display the field when it's set (or pverbose)
  * The possible "keywords" are:
- *  CATEGORY  P  PN  PV  PVR  PF  PR  SLOT  REPO  USE
+ *  CATEGORY  P  PN  PV  PVR  PF  PR  SLOT  SUBSLOT  REPO  USE
  *    - these are all the standard portage variables (so see ebuild(5))
  *  pfx - the version qualifier if set (e.g. > < = !)
  *  sfx - the version qualifier if set (e.g. *)
@@ -695,11 +695,16 @@ atom_format_r(
 								CYAN, atom->PR_int, NORM);
 				} else if (!strncmp("SLOT", fmt, len)) {
 					if (showit || atom->SLOT)
-						append_buf(buf, buflen, "%s%s%s%s%s%s%s",
+						append_buf(buf, buflen, "%s%s%s%s",
 								YELLOW,
 								bracket == '[' ? ":" : "",
 								atom->SLOT ? atom->SLOT : "<unset>",
-								atom->SUBSLOT ? "/" : "",
+								NORM);
+				} else if (!strncmp("SUBSLOT", fmt, len)) {
+					if (showit || atom->SUBSLOT)
+						append_buf(buf, buflen, "%s%s%s%s%s",
+								YELLOW,
+								bracket == '[' ? "/" : "",
 								atom->SUBSLOT ? atom->SUBSLOT : "",
 								atom_slotdep_str[atom->slotdep],
 								NORM);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     4c61b246302353878ea28a90b5b0f4fe5e1e7ca3
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat May 11 06:51:42 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat May 11 06:51:42 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=4c61b246

libq/tree: allow reconstruction of atom in tree_get_atom

if SLOT or REPO got unset in the atom or slot or repo were somehow set
elsewhere, reuse those values to populate the atom with

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 24 ++++++++++++++----------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index 86dd18f..8384458 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -993,24 +993,28 @@ tree_get_atom(tree_pkg_ctx *pkg_ctx, bool complete)
 		tree_ctx *ctx = pkg_ctx->cat_ctx->ctx;
 		if (ctx->cachetype == CACHE_VDB) {
 			if (pkg_ctx->atom->SLOT == NULL) {
-				tree_pkg_vdb_eat(pkg_ctx, "SLOT",
-						&pkg_ctx->slot, &pkg_ctx->slot_len);
+				if (pkg_ctx->slot == NULL)
+					tree_pkg_vdb_eat(pkg_ctx, "SLOT",
+							&pkg_ctx->slot, &pkg_ctx->slot_len);
 				pkg_ctx->atom->SLOT = pkg_ctx->slot;
 			}
 			if (pkg_ctx->atom->REPO == NULL) {
-				tree_pkg_vdb_eat(pkg_ctx, "repository",
-						&pkg_ctx->repo, &pkg_ctx->repo_len);
+				if (pkg_ctx->repo == NULL)
+					tree_pkg_vdb_eat(pkg_ctx, "repository",
+							&pkg_ctx->repo, &pkg_ctx->repo_len);
 				pkg_ctx->atom->REPO = pkg_ctx->repo;
 			}
 		} else { /* metadata or ebuild */
 			if (pkg_ctx->atom->SLOT == NULL) {
-				tree_pkg_meta *meta = tree_pkg_read(pkg_ctx);
-				if (meta != NULL) {
-					pkg_ctx->slot = xstrdup(meta->SLOT);
-					pkg_ctx->slot_len = strlen(pkg_ctx->slot);
-					pkg_ctx->atom->SLOT = pkg_ctx->slot;
-					tree_close_meta(meta);
+				if (pkg_ctx->slot == NULL) {
+					tree_pkg_meta *meta = tree_pkg_read(pkg_ctx);
+					if (meta != NULL) {
+						pkg_ctx->slot = xstrdup(meta->SLOT);
+						pkg_ctx->slot_len = strlen(pkg_ctx->slot);
+						tree_close_meta(meta);
+					}
 				}
+				pkg_ctx->atom->SLOT = pkg_ctx->slot;
 			}
 			/* repo is set from the tree, when found */
 			if (pkg_ctx->atom->REPO == NULL)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     5725e66ffa06bff4edf349e4afb834e66f671b69
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri May 10 12:01:44 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri May 10 12:01:44 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=5725e66f

libq/tree: use tree_get_atom in tree_pkg_metadata

apart from that it is simpler, it also allows for reuse

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index bb7eefa..c8b4b5e 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -821,7 +821,6 @@ tree_pkg_metadata(tree_pkg_ctx *pkg_ctx)
 	size_t len;
 	tree_metadata_xml *ret = NULL;
 	struct elist *emailw = NULL;
-	char buf[_Q_PATH_MAX];
 
 	/* lame @$$ XML parsing, I don't want to pull in a real parser
 	 * library because we only retrieve one element for now: email
@@ -832,13 +831,10 @@ tree_pkg_metadata(tree_pkg_ctx *pkg_ctx)
 	if (ctx->cachetype == CACHE_EBUILD) {
 		fd = openat(pkg_ctx->cat_ctx->fd, "metadata", O_RDONLY | O_CLOEXEC);
 	} else {
-		depend_atom *atom;
-		snprintf(buf, sizeof(buf), "%s/%s",
-				pkg_ctx->cat_ctx->name, pkg_ctx->name);
-		atom = atom_explode(buf);
+		char buf[_Q_PATH_MAX];
+		depend_atom *atom = tree_get_atom(pkg_ctx, false);
 		snprintf(buf, sizeof(buf), "../../%s/%s/metadata.xml",
 				atom->CATEGORY, atom->PN);
-		atom_implode(atom);
 		fd = openat(ctx->tree_fd, buf, O_RDONLY | O_CLOEXEC);
 	}
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     f855d0f4f7c3e6e570a1ad3dc98d737e78996e4a
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri May 10 15:23:25 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri May 10 15:23:25 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=f855d0f4

libq/tree: make pkg sorting based on atom_compare

Using alphasort on pkgs makes little sense because they include version
information that needs careful extraction and matching rules as
implemented by atom_compare.

In order to use atom_compare efficiently, that is, reusing the
atom_explode work done for the elements while running qsort, use
tree_get_atom, which caches the retrieved atom.  Extra bonus is that any
function that retrieves the atom afterwards gets it for free.  This
speeds up significantly apps that need to construct atoms, such as
qkeywords.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/tree.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++--------------
 libq/tree.h |  2 +-
 2 files changed, 51 insertions(+), 16 deletions(-)

diff --git a/libq/tree.c b/libq/tree.c
index c8b4b5e..86dd18f 100644
--- a/libq/tree.c
+++ b/libq/tree.c
@@ -25,6 +25,8 @@
 #include <ctype.h>
 #include <xalloc.h>
 
+static int tree_pkg_compar(const void *l, const void *r);
+
 static tree_ctx *
 tree_open_int(const char *sroot, const char *tdir, bool quiet)
 {
@@ -56,7 +58,7 @@ tree_open_int(const char *sroot, const char *tdir, bool quiet)
 	ctx->do_sort = false;
 	ctx->cat_de = NULL;
 	ctx->catsortfunc = alphasort;
-	ctx->pkgsortfunc = alphasort;
+	ctx->pkgsortfunc = tree_pkg_compar;
 	ctx->repo = NULL;
 	ctx->ebuilddir_ctx = NULL;
 	ctx->ebuilddir_pkg_ctx = NULL;
@@ -208,7 +210,7 @@ tree_open_cat(tree_ctx *ctx, const char *name)
 	cat_ctx->fd = fd;
 	cat_ctx->dir = dir;
 	cat_ctx->ctx = ctx;
-	cat_ctx->pkg_de = NULL;
+	cat_ctx->pkg_ctxs = NULL;
 	return cat_ctx;
 }
 
@@ -260,7 +262,7 @@ tree_close_cat(tree_cat_ctx *cat_ctx)
 	/* closedir() above does this for us: */
 	/* close(ctx->fd); */
 	if (cat_ctx->ctx->do_sort)
-		scandir_free(cat_ctx->pkg_de, cat_ctx->pkg_cnt);
+		free(cat_ctx->pkg_ctxs);
 	free(cat_ctx);
 }
 
@@ -309,30 +311,63 @@ tree_open_pkg(tree_cat_ctx *cat_ctx, const char *name)
 	return pkg_ctx;
 }
 
+static int
+tree_pkg_compar(const void *l, const void *r)
+{
+	tree_pkg_ctx *pl = *(tree_pkg_ctx **)l;
+	tree_pkg_ctx *pr = *(tree_pkg_ctx **)r;
+	depend_atom *al = tree_get_atom(pl, false);
+	depend_atom *ar = tree_get_atom(pr, false);
+
+	switch (atom_compare(al, ar)) {
+		case EQUAL:  return  0;
+		case NEWER:  return -1;
+		case OLDER:  return  1;
+		default:     return strcmp(al->PN, ar->PN);
+	}
+
+	/* unreachable */
+}
+
 static tree_pkg_ctx *
 tree_next_pkg_int(tree_cat_ctx *cat_ctx);
 static tree_pkg_ctx *
 tree_next_pkg_int(tree_cat_ctx *cat_ctx)
 {
 	tree_pkg_ctx *pkg_ctx = NULL;
+	const struct dirent *de;
 
 	if (cat_ctx->ctx->do_sort) {
-		if (cat_ctx->pkg_de == NULL) {
-			cat_ctx->pkg_cnt = scandirat(cat_ctx->fd, ".", &cat_ctx->pkg_de,
-					tree_filter_pkg, cat_ctx->ctx->pkgsortfunc);
+		if (cat_ctx->pkg_ctxs == NULL) {
+			size_t pkg_size = 0;
+			cat_ctx->pkg_ctxs = NULL;
+			cat_ctx->pkg_cnt = 0;
 			cat_ctx->pkg_cur = 0;
-		}
+			while ((de = readdir(cat_ctx->dir)) != NULL) {
+				if (tree_filter_pkg(de) == 0)
+					continue;
+
+				if (cat_ctx->pkg_cnt == pkg_size) {
+					pkg_size += 256;
+					cat_ctx->pkg_ctxs = xrealloc(cat_ctx->pkg_ctxs,
+								sizeof(*cat_ctx->pkg_ctxs) * pkg_size);
+				}
+				pkg_ctx = cat_ctx->pkg_ctxs[cat_ctx->pkg_cnt++] =
+					tree_open_pkg(cat_ctx, de->d_name);
+				if (pkg_ctx == NULL)
+					cat_ctx->pkg_cnt--;
+			}
 
-		while (cat_ctx->pkg_cur < cat_ctx->pkg_cnt) {
-			pkg_ctx =
-				tree_open_pkg(cat_ctx,
-						cat_ctx->pkg_de[cat_ctx->pkg_cur++]->d_name);
-			if (!pkg_ctx)
-				continue;
-			break;
+			if (cat_ctx->ctx->pkgsortfunc != NULL) {
+				qsort(cat_ctx->pkg_ctxs, cat_ctx->pkg_cnt,
+						sizeof(*cat_ctx->pkg_ctxs), cat_ctx->ctx->pkgsortfunc);
+			}
 		}
+
+		pkg_ctx = NULL;
+		if (cat_ctx->pkg_cur < cat_ctx->pkg_cnt)
+			pkg_ctx = cat_ctx->pkg_ctxs[cat_ctx->pkg_cur++];
 	} else {
-		const struct dirent *de;
 		do {
 			de = readdir(cat_ctx->dir);
 			if (!de)

diff --git a/libq/tree.h b/libq/tree.h
index 7f05819..36554be 100644
--- a/libq/tree.h
+++ b/libq/tree.h
@@ -48,7 +48,7 @@ struct tree_cat_ctx {
 	int fd;
 	DIR *dir;
 	tree_ctx *ctx;
-	struct dirent **pkg_de;
+	tree_pkg_ctx **pkg_ctxs;
 	size_t pkg_cnt;
 	size_t pkg_cur;
 };

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     be69812ae6332bbd3a00f00dbdcb928176c3a1dc
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri May 10 13:34:56 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri May 10 13:34:56 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=be69812a

libq/scandirat: support compar being NULL

The specs say that if compar is NULL no sorting happens, so just don't
sort if it is NULL (instead of crashing).

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/scandirat.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libq/scandirat.c b/libq/scandirat.c
index b3d0cfe..f28ba0d 100644
--- a/libq/scandirat.c
+++ b/libq/scandirat.c
@@ -64,7 +64,8 @@ scandirat(int dir_fd, const char *dir, struct dirent ***dirlist,
 	}
 	*dirlist = ret;
 
-	qsort(ret, retlen, sizeof(*ret), (void *)compar);
+	if (compar != NULL)
+		qsort(ret, retlen, sizeof(*ret), (void *)compar);
 
 	/* closes underlying fd */
 	closedir(dirp);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     64628c5c37601544f85785a3b8bb343f495200d8
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May  6 18:01:46 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May  6 18:01:46 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=64628c5c

libq/vdb: add switch to vdb_get_atom to make it cheap/expensive

while at it, ensure repeated calls get the atom for "free"

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/vdb.c | 28 +++++++++++++++++++---------
 libq/vdb.h |  2 +-
 2 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/libq/vdb.c b/libq/vdb.c
index 5dc5e79..810a84c 100644
--- a/libq/vdb.c
+++ b/libq/vdb.c
@@ -414,17 +414,27 @@ next_entry:
 }
 
 depend_atom *
-vdb_get_atom(vdb_pkg_ctx *pkg_ctx)
+vdb_get_atom(vdb_pkg_ctx *pkg_ctx, bool complete)
 {
-	pkg_ctx->atom = atom_explode(pkg_ctx->name);
-	if (pkg_ctx->atom == NULL)
-		return NULL;
-	pkg_ctx->atom->CATEGORY = (char *)pkg_ctx->cat_ctx->name;
+	if (pkg_ctx->atom == NULL) {
+		pkg_ctx->atom = atom_explode(pkg_ctx->name);
+		if (pkg_ctx->atom == NULL)
+			return NULL;
+		pkg_ctx->atom->CATEGORY = (char *)pkg_ctx->cat_ctx->name;
+	}
 
-	vdb_pkg_eat(pkg_ctx, "SLOT", &pkg_ctx->slot, &pkg_ctx->slot_len);
-	pkg_ctx->atom->SLOT = pkg_ctx->slot;
-	vdb_pkg_eat(pkg_ctx, "repository", &pkg_ctx->repo, &pkg_ctx->repo_len);
-	pkg_ctx->atom->REPO = pkg_ctx->repo;
+	if (complete) {
+		if (pkg_ctx->atom->SLOT == NULL) {
+			vdb_pkg_eat(pkg_ctx, "SLOT",
+					&pkg_ctx->slot, &pkg_ctx->slot_len);
+			pkg_ctx->atom->SLOT = pkg_ctx->slot;
+		}
+		if (pkg_ctx->atom->REPO == NULL) {
+			vdb_pkg_eat(pkg_ctx, "repository",
+					&pkg_ctx->repo, &pkg_ctx->repo_len);
+			pkg_ctx->atom->REPO = pkg_ctx->repo;
+		}
+	}
 
 	return pkg_ctx->atom;
 }

diff --git a/libq/vdb.h b/libq/vdb.h
index 2954bef..28ca040 100644
--- a/libq/vdb.h
+++ b/libq/vdb.h
@@ -91,6 +91,6 @@ int vdb_foreach_pkg_sorted(const char *sroot, const char *svdb,
 		vdb_pkg_cb callback, void *priv);
 struct dirent *vdb_get_next_dir(DIR *dir);
 set *get_vdb_atoms(const char *sroot, const char *svdb, int fullcpv);
-depend_atom *vdb_get_atom(vdb_pkg_ctx *pkg_ctx);
+depend_atom *vdb_get_atom(vdb_pkg_ctx *pkg_ctx, bool complete);
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     f837df90e2b14ca2840c1f4280765f5357b9579a
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May  6 16:02:02 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May  6 16:02:02 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=f837df90

libq/vdb: add q_vdb_get_atom to retrieve filled up depend_atom

The atom from q_vdb_get_atom gets freed on q_vdb_close_pkg, and is
filled with SLOT and repository information from VDB.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/vdb.c | 19 +++++++++++++++++++
 libq/vdb.h |  2 ++
 2 files changed, 21 insertions(+)

diff --git a/libq/vdb.c b/libq/vdb.c
index 447bd6f..034a28c 100644
--- a/libq/vdb.c
+++ b/libq/vdb.c
@@ -4,6 +4,7 @@
  *
  * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
  * Copyright 2005-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2019-     Fabian Groffen  - <grobian@gentoo.org>
  */
 
 #include "main.h"
@@ -334,6 +335,8 @@ q_vdb_close_pkg(q_vdb_pkg_ctx *pkg_ctx)
 {
 	if (pkg_ctx->fd != -1)
 		close(pkg_ctx->fd);
+	if (pkg_ctx->atom != NULL)
+		atom_implode(pkg_ctx->atom);
 	free(pkg_ctx->slot);
 	free(pkg_ctx->repo);
 	free(pkg_ctx);
@@ -409,6 +412,22 @@ next_entry:
 	return ret;
 }
 
+depend_atom *
+q_vdb_get_atom(q_vdb_pkg_ctx *pkg_ctx)
+{
+	pkg_ctx->atom = atom_explode(pkg_ctx->name);
+	if (pkg_ctx->atom == NULL)
+		return NULL;
+	pkg_ctx->atom->CATEGORY = (char *)pkg_ctx->cat_ctx->name;
+
+	q_vdb_pkg_eat(pkg_ctx, "SLOT", &pkg_ctx->slot, &pkg_ctx->slot_len);
+	pkg_ctx->atom->SLOT = pkg_ctx->slot;
+	q_vdb_pkg_eat(pkg_ctx, "repository", &pkg_ctx->repo, &pkg_ctx->repo_len);
+	pkg_ctx->atom->REPO = pkg_ctx->repo;
+
+	return pkg_ctx->atom;
+}
+
 set *
 get_vdb_atoms(const char *sroot, const char *svdb, int fullcpv)
 {

diff --git a/libq/vdb.h b/libq/vdb.h
index ee2ee69..3cfa95b 100644
--- a/libq/vdb.h
+++ b/libq/vdb.h
@@ -59,6 +59,7 @@ struct q_vdb_pkg_ctx {
 	size_t repo_len;
 	int fd;
 	q_vdb_cat_ctx *cat_ctx;
+	depend_atom *atom;
 };
 
 /* Global helpers */
@@ -90,5 +91,6 @@ int q_vdb_foreach_pkg_sorted(const char *sroot, const char *svdb,
 		q_vdb_pkg_cb callback, void *priv);
 struct dirent *q_vdb_get_next_dir(DIR *dir);
 set *get_vdb_atoms(const char *sroot, const char *svdb, int fullcpv);
+depend_atom *q_vdb_get_atom(q_vdb_pkg_ctx *pkg_ctx);
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     3253de392836e709cea75f4b3bf3b6e1053c955b
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon May  6 16:01:12 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon May  6 16:01:12 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=3253de39

libq/atom: ignore REPO like SLOT and CATEGORY iff not set on both sides

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 65c8a26..4df76a6 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -406,22 +406,21 @@ atom_compare(const depend_atom *a1, const depend_atom *a2)
 	}
 
 	/* check slot only when both sides have it */
-	if (a1->SLOT && a2->SLOT &&
+	if (a1->SLOT != NULL && a2->SLOT != NULL &&
 			a1->SLOT[0] != '\0' && a2->SLOT[0] != '\0' &&
 			(strcmp(a1->SLOT, a2->SLOT) != 0 ||
 			(a1->SUBSLOT != NULL && a2->SUBSLOT != NULL &&
 			 strcmp(a1->SUBSLOT, a2->SUBSLOT) != 0)))
 		return NOT_EQUAL;
 
-	/* check repo */
-	if (a1->REPO || a2->REPO) {
-		if (!a1->REPO || !a2->REPO || strcmp(a1->REPO, a2->REPO))
-			return NOT_EQUAL;
-	}
+	/* same for check repo */
+	if (a1->REPO != NULL && a2->REPO != NULL &&
+			a1->REPO[0] != '\0' && a2->REPO[0] != '\0' &&
+			strcmp(a1->REPO, a2->REPO) != 0)
+		return NOT_EQUAL;
 
 	/* Check category, iff both are specified.  This way we can match
-	 * atoms like "sys-devel/gcc" and "gcc".
-	 */
+	 * atoms like "sys-devel/gcc" and "gcc". */
 	if (a1->CATEGORY && a2->CATEGORY) {
 		if (strcmp(a1->CATEGORY, a2->CATEGORY))
 			return NOT_EQUAL;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     ed9eec8743923166205a4fdc7f9bb073ad6c5ba0
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun May  5 20:04:07 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun May  5 20:04:07 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ed9eec87

libq/atom: add colour to formatted atoms

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 53 ++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 38 insertions(+), 15 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 8eb9ac6..65c8a26 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -610,7 +610,7 @@ atom_to_string_r(char *buf, size_t buflen, depend_atom *a)
  *  %{keyword}: Always display the field that matches "keyword" or <unset>
  *  %[keyword]: Only display the field when it's set (or pverbose)
  * The possible "keywords" are:
- *  CATEGORY  P  PN  PV  PVR  PF  PR  SLOT
+ *  CATEGORY  P  PN  PV  PVR  PF  PR  SLOT  REPO  USE
  *    - these are all the standard portage variables (so see ebuild(5))
  *  pfx - the version qualifier if set (e.g. > < = !)
  *  sfx - the version qualifier if set (e.g. *)
@@ -663,39 +663,52 @@ atom_format_r(
 #define HN(X) (X ? X : "<unset>")
 				if (!strncmp("CATEGORY", fmt, len)) {
 					if (showit || atom->CATEGORY)
-						append_buf(buf, buflen, "%s", HN(atom->CATEGORY));
+						append_buf(buf, buflen, "%s%s%s%s",
+								BOLD, HN(atom->CATEGORY),
+								bracket == '[' ? "/" : "", NORM);
 				} else if (!strncmp("P", fmt, len)) {
 					if (showit || atom->P)
-						append_buf(buf, buflen, "%s", HN(atom->P));
+						append_buf(buf, buflen, "%s%s%s",
+								BLUE, HN(atom->P), NORM);
 				} else if (!strncmp("PN", fmt, len)) {
 					if (showit || atom->PN)
-						append_buf(buf, buflen, "%s", HN(atom->PN));
+						append_buf(buf, buflen, "%s%s%s",
+								BLUE, HN(atom->PN), NORM);
 				} else if (!strncmp("PV", fmt, len)) {
 					if (showit || atom->PV)
-						append_buf(buf, buflen, "%s", HN(atom->PV));
+						append_buf(buf, buflen, "%s%s%s",
+								DKBLUE, HN(atom->PV), NORM);
 				} else if (!strncmp("PVR", fmt, len)) {
 					if (showit || atom->PVR)
-						append_buf(buf, buflen, "%s", HN(atom->PVR));
+						append_buf(buf, buflen, "%s%s%s",
+								DKBLUE, HN(atom->PVR), NORM);
 				} else if (!strncmp("PF", fmt, len)) {
-					append_buf(buf, buflen, "%s", atom->PN);
+					append_buf(buf, buflen, "%s%s%s", BLUE, atom->PN, NORM);
 					if (atom->PV)
-						append_buf(buf, buflen, "-%s", atom->PV);
+						append_buf(buf, buflen, "%s-%s%s",
+								DKBLUE, atom->PV, NORM);
 					if (atom->PR_int)
-						append_buf(buf, buflen,"-r%i", atom->PR_int);
+						append_buf(buf, buflen,"%s-r%d%s",
+								DKBLUE, atom->PR_int, NORM);
 				} else if (!strncmp("PR", fmt, len)) {
 					if (showit || atom->PR_int)
-						append_buf(buf, buflen, "r%i", atom->PR_int);
+						append_buf(buf, buflen, "%sr%d%s",
+								DKBLUE, atom->PR_int, NORM);
 				} else if (!strncmp("SLOT", fmt, len)) {
 					if (showit || atom->SLOT)
-						append_buf(buf, buflen, "%s%s%s%s%s",
-								atom->SLOT ? ":" : "<unset>",
-								atom->SLOT ? atom->SLOT : "",
+						append_buf(buf, buflen, "%s%s%s%s%s%s%s",
+								YELLOW,
+								bracket == '[' ? ":" : "",
+								atom->SLOT ? atom->SLOT : "<unset>",
 								atom->SUBSLOT ? "/" : "",
 								atom->SUBSLOT ? atom->SUBSLOT : "",
-								atom_slotdep_str[atom->slotdep]);
+								atom_slotdep_str[atom->slotdep],
+								NORM);
 				} else if (!strncmp("REPO", fmt, len)) {
 					if (showit || atom->REPO)
-						append_buf(buf, buflen, "::%s", HN(atom->REPO));
+						append_buf(buf, buflen, "%s%s%s%s",
+								GREEN, bracket == '[' ? "::" : "",
+								HN(atom->REPO), NORM);
 				} else if (!strncmp("pfx", fmt, len)) {
 					if (showit || atom->pfx_op != ATOM_OP_NONE)
 						append_buf(buf, buflen, "%s",
@@ -706,6 +719,16 @@ atom_format_r(
 						append_buf(buf, buflen, "%s",
 								atom->sfx_op == ATOM_OP_NONE ?
 								"<unset>" : atom_op_str[atom->sfx_op]);
+				} else if (!strncmp("USE", fmt, len)) {
+					if (showit || atom->usedeps) {
+						atom_usedep *ud;
+						append_buf(buf, buflen, "%s", "[");
+						for (ud = atom->usedeps; ud != NULL; ud = ud->next)
+							append_buf(buf, buflen, "%s%s%s%s%s%s",
+									MAGENTA, atom_usecond_str[ud->pfx_cond],
+									ud->use, atom_usecond_str[ud->sfx_cond],
+									NORM, ud->next == NULL ? "]" : ",");
+					}
 				} else
 					append_buf(buf, buflen, "<BAD:%.*s>", (int)len, fmt);
 				++p;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     caa404364ae6e08ea2af6bbee7a6b0432162edef
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun May  5 17:06:00 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun May  5 17:06:00 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=caa40436

libq/atom: drop non-functional USE_CACHE bits

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 37 +++----------------------------------
 1 file changed, 3 insertions(+), 34 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 75a621f..5f4b137 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -38,27 +38,6 @@ const char * const atom_op_str[] = {
 
 const char * const booga[] = {"!!!", "!=", "==", ">", "<"};
 
-#ifdef EBUG
-void
-atom_print(const depend_atom *atom)
-{
-	if (atom->CATEGORY)
-		printf("%s/", atom->CATEGORY);
-	printf("%s", atom->P);
-	if (atom->PR_int)
-		printf("-r%i", atom->PR_int);
-	if (atom->SLOT)
-		printf(":%s", atom->SLOT);
-	if (atom->REPO)
-		printf("::%s", atom->REPO);
-}
-#endif
-
-#ifdef _USE_CACHE
-static depend_atom *_atom_cache = NULL;
-static size_t _atom_cache_len = 0;
-#endif
-
 depend_atom *
 atom_explode(const char *atom)
 {
@@ -74,18 +53,7 @@ atom_explode(const char *atom)
 	 * PVR needs 3 extra bytes for possible implicit '-r0'. */
 	slen = strlen(atom);
 	len = sizeof(*ret) + (slen + 1) * sizeof(*atom) * 3 + 3;
-#ifdef _USE_CACHE
-	if (len <= _atom_cache_len) {
-		ret = _atom_cache;
-		memset(ret, 0x00, len);
-	} else {
-		free(_atom_cache);
-		_atom_cache = ret = xzalloc(len);
-		_atom_cache_len = len;
-	}
-#else
 	ret = xzalloc(len);
-#endif
 	ptr = (char*)ret;
 	ret->P = ptr + sizeof(*ret);
 	ret->PVR = ret->P + slen + 1;
@@ -356,9 +324,7 @@ atom_implode(depend_atom *atom)
 	if (!atom)
 		errf("Atom is empty !");
 	free(atom->suffixes);
-#ifndef _USE_CACHE
 	free(atom);
-#endif
 }
 
 static int
@@ -600,6 +566,9 @@ implode_a1_ret:
 	return ret;
 }
 
+/**
+ * Reconstructs an atom exactly like it was originally given (exploded).
+ */
 static char _atom_buf[BUFSIZ];
 char *
 atom_to_string(depend_atom *a)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     0bbdb3f1328dcd0dd91c1ea502da3ba3e1fd364d
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun May  5 08:46:47 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun May  5 08:46:47 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=0bbdb3f1

libq/cache: add support for reading metadata.xml

currently only one field is extracted: email

Bug: https://bugs.gentoo.org/685052
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 libq/cache.h |  9 ++++++
 2 files changed, 106 insertions(+), 2 deletions(-)

diff --git a/libq/cache.c b/libq/cache.c
index a8b4ede..c0ea85e 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -167,6 +167,9 @@ cache_next_pkg(cache_cat_ctx *cat_ctx)
 					ctx->ebuilddir_pkg_ctx = NULL;
 					return NULL;
 				}
+
+				/* "zap" the pkg such that it looks like CAT/P */
+				ctx->ebuilddir_cat_ctx->name = cat_ctx->name;
 			}
 
 			ret = q_vdb_next_pkg(ctx->ebuilddir_cat_ctx);
@@ -178,8 +181,6 @@ cache_next_pkg(cache_cat_ctx *cat_ctx)
 					cache_close_pkg(ret);
 					ret = NULL;
 				} else {
-					/* "zap" the pkg such that it looks like CAT/P */
-					ret->cat_ctx->name = cat_ctx->name;
 					*p = '\0';
 				}
 			}
@@ -529,6 +530,100 @@ cache_close_meta(cache_pkg_meta *cache)
 	free(cache);
 }
 
+cache_metadata_xml *
+cache_read_metadata(cache_pkg_ctx *pkg_ctx)
+{
+	cache_ctx *ctx = (cache_ctx *)(pkg_ctx->cat_ctx->ctx);
+	int fd;
+	FILE *f;
+	struct stat s;
+	char *xbuf;
+	char *p;
+	char *q;
+	size_t len;
+	cache_metadata_xml *ret = NULL;
+	struct elist *emailw = NULL;
+	char buf[_Q_PATH_MAX];
+
+	/* lame @$$ XML parsing, I don't want to pull in a real parser
+	 * library because we only retrieve one element for now: email
+	 * technically speaking, email may occur only once in a maintainer
+	 * tag, but practically speaking we don't care at all, so we can
+	 * just extract everything between <email> and </email> */
+
+	if (ctx->cachetype == CACHE_EBUILD) {
+		fd = openat(pkg_ctx->cat_ctx->fd, "metadata", O_RDONLY | O_CLOEXEC);
+	} else {
+		depend_atom *atom;
+		snprintf(buf, sizeof(buf), "%s/%s",
+				pkg_ctx->cat_ctx->name, pkg_ctx->name);
+		atom = atom_explode(buf);
+		snprintf(buf, sizeof(buf), "../../%s/%s/metadata.xml",
+				atom->CATEGORY, atom->PN);
+		atom_implode(atom);
+		fd = openat(ctx->vdb_fd, buf, O_RDONLY | O_CLOEXEC);
+	}
+
+	if (fd == -1)
+		return NULL;
+
+	if ((f = fdopen(fd, "r")) == NULL) {
+		close(fd);
+		return NULL;
+	}
+
+	if (fstat(fd, &s) != 0) {
+		fclose(f);
+		return NULL;
+	}
+
+	len = sizeof(*ret) + s.st_size + 1;
+	p = xbuf = xzalloc(len);
+	if ((off_t)fread(p, 1, s.st_size, f) != s.st_size) {
+		free(p);
+		fclose(f);
+		pkg_ctx->fd = -1;
+		return NULL;
+	}
+
+	ret = xmalloc(sizeof(*ret));
+	ret->email = NULL;
+
+	while ((q = strstr(p, "<email>")) != NULL) {
+		p = q + sizeof("<email>") - 1;
+		if ((q = strstr(p, "</email>")) == NULL)
+			break;
+		*q = '\0';
+		rmspace(p);
+		if (emailw == NULL) {
+			emailw = ret->email = xmalloc(sizeof(*emailw));
+		} else {
+			emailw = emailw->next = xmalloc(sizeof(*emailw));
+		}
+		emailw->next = NULL;
+		emailw->addr = xstrdup(p);
+		p = q + 1;
+	}
+
+	free(xbuf);
+	fclose(f);
+	return ret;
+}
+
+void
+cache_close_metadata(cache_metadata_xml *meta_ctx)
+{
+	struct elist *e;
+	while (meta_ctx->email != NULL) {
+		e = meta_ctx->email;
+		free(e->addr);
+		e = e->next;
+		free(meta_ctx->email);
+		meta_ctx->email = e;
+	}
+	free(meta_ctx);
+}
+
 void
 cache_close_pkg(cache_pkg_ctx *pkg_ctx)
 {

diff --git a/libq/cache.h b/libq/cache.h
index 74f45b8..2ad2e78 100644
--- a/libq/cache.h
+++ b/libq/cache.h
@@ -43,6 +43,13 @@ typedef struct {
 	char *_md5_;
 } cache_pkg_meta;
 
+typedef struct {
+	struct elist {
+		char *addr;
+		struct elist *next;
+	} *email;
+} cache_metadata_xml;
+
 typedef int (cache_pkg_cb)(cache_pkg_ctx *, void *priv);
 typedef int (cache_cat_filter)(cache_cat_ctx *, void *priv);
 
@@ -55,6 +62,8 @@ cache_pkg_ctx *cache_open_pkg(cache_cat_ctx *cat_ctx, const char *name);
 cache_pkg_ctx *cache_next_pkg(cache_cat_ctx *cat_ctx);
 cache_pkg_meta *cache_pkg_read(cache_pkg_ctx *pkg_ctx);
 void cache_close_meta(cache_pkg_meta *cache);
+cache_metadata_xml *cache_read_metadata(cache_pkg_ctx *pkg_ctx);
+void cache_close_metadata(cache_metadata_xml *meta_ctx);
 void cache_close_pkg(cache_pkg_ctx *pkg_ctx);
 int cache_foreach_pkg(const char *sroot, const char *portdir,
 		cache_pkg_cb callback, void *priv, cache_cat_filter filter);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     55a3b37772e4e4f70b3eaf76916f2b1450d26a6e
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat May  4 11:51:31 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat May  4 11:51:31 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=55a3b377

libq/cache: probably fix crash from bug #684252#c22

Ensure we renew the local ebuild dir context after we found a problem
with the previous directory

Bug: https://bugs.gentoo.org/684252
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/libq/cache.c b/libq/cache.c
index 0dbf483..a8b4ede 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -163,8 +163,10 @@ cache_next_pkg(cache_cat_ctx *cat_ctx)
 
 				/* opening might fail if what we found wasn't a
 				 * directory or something */
-				if (ctx->ebuilddir_cat_ctx == NULL)
+				if (ctx->ebuilddir_cat_ctx == NULL) {
+					ctx->ebuilddir_pkg_ctx = NULL;
 					return NULL;
+				}
 			}
 
 			ret = q_vdb_next_pkg(ctx->ebuilddir_cat_ctx);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     ab51b5e60db69f6ee6f2513fef7ce12ebc01af92
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri May  3 11:44:40 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri May  3 11:44:40 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ab51b5e6

libq/cache: fix crash when a category contains a non-directory

Bug: https://bugs.gentoo.org/684252
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libq/cache.c b/libq/cache.c
index 9993002..0dbf483 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -160,6 +160,11 @@ cache_next_pkg(cache_cat_ctx *cat_ctx)
 
 				ctx->ebuilddir_cat_ctx =
 					q_vdb_open_cat(pkgdir, ctx->ebuilddir_pkg_ctx->name);
+
+				/* opening might fail if what we found wasn't a
+				 * directory or something */
+				if (ctx->ebuilddir_cat_ctx == NULL)
+					return NULL;
 			}
 
 			ret = q_vdb_next_pkg(ctx->ebuilddir_cat_ctx);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     b6866cb7cbe9172ad2ac46e5e72b71901ec44841
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu May  2 15:16:16 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu May  2 15:16:16 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=b6866cb7

libq/cache: fix handling of (escaped) quotes in cache_read_file_ebuild

Bug: https://bugs.gentoo.org/684252
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/libq/cache.c b/libq/cache.c
index b11e068..ee3a47c 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -429,8 +429,11 @@ cache_read_file_ebuild(cache_pkg_ctx *pkg_ctx)
 						for (r = p - 1; r > q; r--)
 							if (*r != '\\')
 								break;
-						if (r != q && (p - 1 - r) % 2 == 1)
+						if (r != q && (p - 1 - r) % 2 == 1) {
+							/* escaped, move along */
+							p++;
 							continue;
+						}
 					}
 					break;
 				} while (1);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     cca8e87817039f6c536304799cae364b264a75f4
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed May  1 19:09:19 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed May  1 19:09:19 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=cca8e878

libq/cache: ensure ebuild traversal gets correct sort funcs

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libq/cache.c b/libq/cache.c
index ab7e4f8..b11e068 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -153,6 +153,8 @@ cache_next_pkg(cache_cat_ctx *cat_ctx)
 				pkgdir->portroot_fd = -1;
 				pkgdir->vdb_fd = cat_ctx->fd;
 				pkgdir->do_sort = ctx->do_sort;
+				pkgdir->catsortfunc = ctx->catsortfunc;
+				pkgdir->pkgsortfunc = ctx->pkgsortfunc;
 				pkgdir->repo = ctx->repo;
 				pkgdir->cachetype = ctx->cachetype;
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     fad4e2f0ce2310f180c8cd25902aa1ee4de44390
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Apr 30 08:19:52 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue Apr 30 08:19:52 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=fad4e2f0

libq/cache: fix reading quoted strings in cache_read_file_ebuild

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/cache.c b/libq/cache.c
index 504b6d3..ab7e4f8 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -419,7 +419,7 @@ cache_read_file_ebuild(cache_pkg_ctx *pkg_ctx)
 			q = p;
 			if (*q == '"' || *q == '\'') {
 				/* find matching quote */
-				q = p;
+				p++;
 				do {
 					while (*p != '\0' && *p != *q)
 						p++;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     737cbc7da8473b7f2cd98b18201bf438c7e6c8c7
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Apr 30 07:52:27 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue Apr 30 07:52:27 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=737cbc7d

libq/cache: fix some issues (likely bugs #684252 and #684476)

cache_next_pkg: fix package traversal and cat_ctx shoveling
cache_read_file_ebuild: fix key parsing (not to loop forever)
cache_pkg_read: open correct file for ebuild case

Bug: https://bugs.gentoo.org/684252
Bug: https://bugs.gentoo.org/684476
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 37 ++++++++++++++++++++++++++++---------
 1 file changed, 28 insertions(+), 9 deletions(-)

diff --git a/libq/cache.c b/libq/cache.c
index efc47bc..504b6d3 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -144,14 +144,17 @@ cache_next_pkg(cache_cat_ctx *cat_ctx)
 				q_vdb_ctx *pkgdir = ctx->ebuilddir_ctx;
 
 				if (pkgdir == NULL)
-					pkgdir = ctx->ebuilddir_ctx = xzalloc(sizeof(q_vdb_ctx));
+					pkgdir = ctx->ebuilddir_ctx = xmalloc(sizeof(q_vdb_ctx));
+				memset(ctx->ebuilddir_ctx, '\0', sizeof(*ctx->ebuilddir_ctx));
 
 				if ((ctx->ebuilddir_pkg_ctx = q_vdb_next_pkg(cat_ctx)) == NULL)
 					return NULL;
 
 				pkgdir->portroot_fd = -1;
 				pkgdir->vdb_fd = cat_ctx->fd;
-				pkgdir->dir = NULL;
+				pkgdir->do_sort = ctx->do_sort;
+				pkgdir->repo = ctx->repo;
+				pkgdir->cachetype = ctx->cachetype;
 
 				ctx->ebuilddir_cat_ctx =
 					q_vdb_open_cat(pkgdir, ctx->ebuilddir_pkg_ctx->name);
@@ -159,6 +162,7 @@ cache_next_pkg(cache_cat_ctx *cat_ctx)
 
 			ret = q_vdb_next_pkg(ctx->ebuilddir_cat_ctx);
 			if (ret == NULL) {
+				q_vdb_close_cat(ctx->ebuilddir_cat_ctx);
 				ctx->ebuilddir_pkg_ctx = NULL;
 			} else {
 				if ((p = strstr(ret->name, ".ebuild")) == NULL) {
@@ -166,7 +170,7 @@ cache_next_pkg(cache_cat_ctx *cat_ctx)
 					ret = NULL;
 				} else {
 					/* "zap" the pkg such that it looks like CAT/P */
-					ret->cat_ctx = cat_ctx;
+					ret->cat_ctx->name = cat_ctx->name;
 					*p = '\0';
 				}
 			}
@@ -366,6 +370,7 @@ cache_read_file_ebuild(cache_pkg_ctx *pkg_ctx)
 	char *q;
 	char *r;
 	char **key;
+	bool findnl;
 
 	if ((f = fdopen(pkg_ctx->fd, "r")) == NULL)
 		goto err;
@@ -382,8 +387,6 @@ cache_read_file_ebuild(cache_pkg_ctx *pkg_ctx)
 
 	p = ret->_data;
 	do {
-		if ((p = strchr(p, '\n')) == NULL)
-			break;
 		q = p;
 		while (*p >= 'A' && *p <= 'Z')
 			p++;
@@ -411,11 +414,12 @@ cache_read_file_ebuild(cache_pkg_ctx *pkg_ctx)
 #undef match_key
 		}
 
+		findnl = true;
 		if (key != NULL) {
 			q = p;
 			if (*q == '"' || *q == '\'') {
 				/* find matching quote */
-				q = ++p;
+				q = p;
 				do {
 					while (*p != '\0' && *p != *q)
 						p++;
@@ -428,14 +432,20 @@ cache_read_file_ebuild(cache_pkg_ctx *pkg_ctx)
 					}
 					break;
 				} while (1);
+				q++;
 			} else {
 				/* find first whitespace */
 				while (!isspace((int)*p))
 					p++;
+				if (*p == '\n')
+					findnl = false;
 			}
-			*p = '\0';
+			*p++ = '\0';
 			*key = q;
 		}
+
+		if (findnl && (p = strchr(p, '\n')) != NULL)
+			p++;
 	} while (p != NULL);
 
 	fclose(f);
@@ -458,8 +468,17 @@ cache_pkg_read(cache_pkg_ctx *pkg_ctx)
 	cache_ctx *ctx = (cache_ctx *)(pkg_ctx->cat_ctx->ctx);
 
 	if (pkg_ctx->fd == -1) {
-		pkg_ctx->fd = openat(pkg_ctx->cat_ctx->fd, pkg_ctx->name,
-				O_RDONLY|O_CLOEXEC);
+		if (ctx->cachetype != CACHE_EBUILD) {
+			pkg_ctx->fd = openat(pkg_ctx->cat_ctx->fd, pkg_ctx->name,
+					O_RDONLY|O_CLOEXEC);
+		} else {
+			char *p = (char *)pkg_ctx->name;
+			p += strlen(p);
+			*p = '.';
+			pkg_ctx->fd = openat(pkg_ctx->cat_ctx->fd, pkg_ctx->name,
+					O_RDONLY|O_CLOEXEC);
+			*p = '\0';
+		}
 		if (pkg_ctx->fd == -1)
 			return NULL;
 	}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     8b633323765e129644ad9a4d93e0d6c467927aa4
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Apr 28 16:49:37 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Apr 28 17:09:51 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=8b633323

libq/vdb: implement q_vdb_filter_pkg following PMS

exclude things like Metadata.gz and other random cruft by following PMS
package name descriptions

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/vdb.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/libq/vdb.c b/libq/vdb.c
index 34edfd2..447bd6f 100644
--- a/libq/vdb.c
+++ b/libq/vdb.c
@@ -88,6 +88,7 @@ q_vdb_filter_cat(const struct dirent *de)
 	bool founddash;
 
 #ifdef DT_UNKNOWN
+	/* cat must be a dir */
 	if (de->d_type != DT_UNKNOWN &&
 	    de->d_type != DT_DIR &&
 	    de->d_type != DT_LNK)
@@ -204,10 +205,33 @@ q_vdb_close_cat(q_vdb_cat_ctx *cat_ctx)
 int
 q_vdb_filter_pkg(const struct dirent *de)
 {
-	if (de->d_name[0] == '.' || de->d_name[0] == '-')
-		return 0;
+	int i;
+	bool founddash = false;
 
-	return 1;
+	/* PMS 3.1.2 */
+	for (i = 0; de->d_name[i] != '\0'; i++) {
+		switch (de->d_name[i]) {
+			case '_':
+				break;
+			case '-':
+				founddash = true;
+				/* fall through */
+			case '+':
+				if (i)
+					break;
+				return 0;
+			default:
+				if ((de->d_name[i] >= 'A' && de->d_name[i] <= 'Z') ||
+						(de->d_name[i] >= 'a' && de->d_name[i] <= 'z') ||
+						(de->d_name[i] >= '0' && de->d_name[i] <= '9'))
+					break;
+				if (founddash)
+					return 1;
+				return 0;
+		}
+	}
+
+	return i;
 }
 
 q_vdb_pkg_ctx *

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e5251bd75fe11b3a350d122c29a994f54f069974
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Apr 28 16:21:01 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Apr 28 16:21:01 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e5251bd7

libq/cache: support BDEPEND and fix error messages

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 16 ++++++++++------
 libq/cache.h |  1 +
 2 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/libq/cache.c b/libq/cache.c
index a00dd6b..efc47bc 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -255,7 +255,6 @@ cache_read_file_md5(cache_pkg_ctx *pkg_ctx)
 	FILE *f;
 	cache_pkg_meta *ret = NULL;
 	size_t len;
-	char buf[_Q_PATH_MAX];
 
 	if ((f = fdopen(pkg_ctx->fd, "r")) == NULL)
 		goto err;
@@ -288,8 +287,9 @@ cache_read_file_md5(cache_pkg_ctx *pkg_ctx)
 	ptr = ret->_data;
 	endptr = strchr(ptr, '\0');
 	if (endptr == NULL) {
-			warn("Invalid cache file for '%s': "
-					"could not find end of cache data", buf);
+			warn("Invalid cache file for '%s/%s': "
+					"could not find end of cache data",
+					pkg_ctx->cat_ctx->name, pkg_ctx->name);
 			goto err;
 	}
 
@@ -299,14 +299,16 @@ cache_read_file_md5(cache_pkg_ctx *pkg_ctx)
 		keyptr = ptr;
 		valptr = strchr(ptr, '=');
 		if (valptr == NULL) {
-			warn("Invalid cache file for '%s': missing val", buf);
+			warn("Invalid cache file for '%s/%s': missing val",
+					pkg_ctx->cat_ctx->name, pkg_ctx->name);
 			goto err;
 		}
 		*valptr = '\0';
 		valptr++;
 		ptr = strchr(valptr, '\n');
 		if (ptr == NULL) {
-			warn("Invalid cache file for '%s': missing key", buf);
+			warn("Invalid cache file for '%s/%s': missing key",
+					pkg_ctx->cat_ctx->name, pkg_ctx->name);
 			goto err;
 		}
 		*ptr = '\0';
@@ -330,9 +332,11 @@ cache_read_file_md5(cache_pkg_ctx *pkg_ctx)
 		assign_var(SRC_URI);
 		assign_var(DEFINED_PHASES);
 		assign_var(REQUIRED_USE);
+		assign_var(BDEPEND);
 		assign_var(_eclasses_);
 		assign_var(_md5_);
-		warn("Cache file for '%s' has unknown key %s", buf, keyptr);
+		warn("Cache file for '%s/%s' has unknown key %s",
+				pkg_ctx->cat_ctx->name, pkg_ctx->name, keyptr);
 	}
 #undef assign_var
 #undef assign_var_cmp

diff --git a/libq/cache.h b/libq/cache.h
index f9b1d43..74f45b8 100644
--- a/libq/cache.h
+++ b/libq/cache.h
@@ -38,6 +38,7 @@ typedef struct {
 	/* These are MD5-Cache only */
 	char *DEFINED_PHASES;
 	char *REQUIRED_USE;
+	char *BDEPEND;
 	char *_eclasses_;
 	char *_md5_;
 } cache_pkg_meta;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     baa5e20fa9d84daa6f12c1cb42c01b4c292092c2
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Apr 28 16:01:43 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Apr 28 16:01:43 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=baa5e20f

libq/vdb: make q_vdb_filter_pkg work on Linux

skip the entry type checks, just look at the name

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/vdb.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/libq/vdb.c b/libq/vdb.c
index 9c53b5b..34edfd2 100644
--- a/libq/vdb.c
+++ b/libq/vdb.c
@@ -204,13 +204,6 @@ q_vdb_close_cat(q_vdb_cat_ctx *cat_ctx)
 int
 q_vdb_filter_pkg(const struct dirent *de)
 {
-#ifdef DT_UNKNOWN
-	if (de->d_type != DT_UNKNOWN &&
-	    de->d_type != DT_DIR &&
-	    de->d_type != DT_LNK)
-		return 0;
-#endif
-
 	if (de->d_name[0] == '.' || de->d_name[0] == '-')
 		return 0;
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     6abbc83f08aaea6ac17c6f68f674d52da596c34d
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Apr 27 08:38:00 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Apr 27 08:38:00 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=6abbc83f

libq/atom: fix whitespace

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libq/atom.c b/libq/atom.c
index 5ded0d9..75a621f 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -148,7 +148,7 @@ atom_explode(const char *atom)
 			ret->slotdep = ATOM_SD_ANY_IGNORE;
 			*ptr = '\0';
 		}
-		
+
 		/* cut in two when sub-slot */
 		if ((ptr = strchr(ret->SLOT, '/')) != NULL) {
 			*ptr++ = '\0';

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     1ff4f0dd61135b86fe5d71e1cc584220b696783b
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Apr 25 17:17:21 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Apr 25 17:17:21 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=1ff4f0dd

libq/atom: support/parse SUBSLOT

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 16 +++++++++++++---
 libq/atom.h |  1 +
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 58565bc..5ded0d9 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -148,6 +148,12 @@ atom_explode(const char *atom)
 			ret->slotdep = ATOM_SD_ANY_IGNORE;
 			*ptr = '\0';
 		}
+		
+		/* cut in two when sub-slot */
+		if ((ptr = strchr(ret->SLOT, '/')) != NULL) {
+			*ptr++ = '\0';
+			ret->SUBSLOT = ptr;
+		}
 	}
 
 	/* see if we have any suffix operators */
@@ -435,7 +441,9 @@ atom_compare(const depend_atom *a1, const depend_atom *a2)
 	/* check slot only when both sides have it */
 	if (a1->SLOT && a2->SLOT &&
 			a1->SLOT[0] != '\0' && a2->SLOT[0] != '\0' &&
-			strcmp(a1->SLOT, a2->SLOT) != 0)
+			(strcmp(a1->SLOT, a2->SLOT) != 0 ||
+			(a1->SUBSLOT != NULL && a2->SUBSLOT != NULL &&
+			 strcmp(a1->SUBSLOT, a2->SUBSLOT) != 0)))
 		return NOT_EQUAL;
 
 	/* check repo */
@@ -620,8 +628,10 @@ atom_to_string(depend_atom *a)
 				atom_usecond_str[ud->sfx_cond],
 				ud->next == NULL ? "]" : ",");
 	if (a->SLOT != NULL)
-		off += snprintf(buf + off, buflen - off, ":%s%s",
-				a->SLOT, atom_slotdep_str[a->slotdep]);
+		off += snprintf(buf + off, buflen - off, ":%s%s%s%s",
+				a->SLOT,
+				a->SUBSLOT ? "/" : "", a->SUBSLOT ? a->SUBSLOT : "",
+				atom_slotdep_str[a->slotdep]);
 	if (a->REPO != NULL)
 		off += snprintf(buf + off, buflen - off, "::%s", a->REPO);
 

diff --git a/libq/atom.h b/libq/atom.h
index c9a1ddb..291d637 100644
--- a/libq/atom.h
+++ b/libq/atom.h
@@ -80,6 +80,7 @@ typedef struct {
 	char *P;
 	atom_usedep *usedeps;
 	char *SLOT;
+	char *SUBSLOT;
 	atom_slotdep slotdep;
 	char *REPO;
 } depend_atom;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     b9329cff22e3e88bd9ba3d259ed586b75fe9740f
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Apr 25 08:46:09 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Apr 25 08:46:09 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=b9329cff

libq/cache: set repo in cache_pkg_ctx when found

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 31 +++++++++++++++++++++++++------
 libq/cache.h |  1 +
 2 files changed, 26 insertions(+), 6 deletions(-)

diff --git a/libq/cache.c b/libq/cache.c
index eec39db..7724287 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -16,6 +16,8 @@
 #include <xalloc.h>
 
 #include "cache.h"
+#include "eat_file.h"
+#include "rmspace.h"
 #include "scandirat.h"
 #include "vdb.h"
 
@@ -47,17 +49,27 @@ cache_dump(cache_pkg_meta *cache)
 
 static const char portcachedir_pms[] = "metadata/cache";
 static const char portcachedir_md5[] = "metadata/md5-cache";
+static const char portrepo_name[]    = "profiles/repo_name";
 cache_ctx *
 cache_open(const char *sroot, const char *portdir)
 {
 	q_vdb_ctx *dir;
 	cache_ctx *ret;
 	char buf[_Q_PATH_MAX];
+	size_t repolen = 0;
+
+	ret = xzalloc(sizeof(cache_ctx));
+
+	snprintf(buf, sizeof(buf), "%s%s/%s", sroot, portdir, portrepo_name);
+	if (eat_file(buf, &ret->repo, &repolen)) {
+		(void)rmspace(ret->repo);
+	} else {
+		ret->repo = NULL;  /* ignore missing repo file */
+	}
 
 	snprintf(buf, sizeof(buf), "%s/%s", portdir, portcachedir_md5);
 	dir = q_vdb_open2(sroot, buf, true);
 	if (dir != NULL) {
-		ret = xmalloc(sizeof(cache_ctx));
 		ret->dir_ctx = dir;
 		ret->cachetype = CACHE_METADATA_MD5;
 		return ret;
@@ -66,7 +78,6 @@ cache_open(const char *sroot, const char *portdir)
 	snprintf(buf, sizeof(buf), "%s/%s", portdir, portcachedir_pms);
 	dir = q_vdb_open2(sroot, buf, true);
 	if (dir != NULL) {
-		ret = xmalloc(sizeof(cache_ctx));
 		ret->dir_ctx = dir;
 		ret->cachetype = CACHE_METADATA_PMS;
 		return ret;
@@ -74,12 +85,12 @@ cache_open(const char *sroot, const char *portdir)
 
 	dir = q_vdb_open2(sroot, portdir, true);
 	if (dir != NULL) {
-		ret = xmalloc(sizeof(cache_ctx));
 		ret->dir_ctx = dir;
 		ret->cachetype = CACHE_EBUILD;
 		return ret;
 	}
 
+	cache_close(ret);
 	warnf("could not open repository at %s (under root %s)",
 			portdir, sroot);
 
@@ -89,7 +100,10 @@ cache_open(const char *sroot, const char *portdir)
 void
 cache_close(cache_ctx *ctx)
 {
-	q_vdb_close(ctx->dir_ctx);
+	if (ctx->dir_ctx != NULL)
+		q_vdb_close(ctx->dir_ctx);
+	if (ctx->repo != NULL)
+		free(ctx->repo);
 	free(ctx);
 }
 
@@ -120,7 +134,9 @@ cache_close_cat(cache_cat_ctx *cat_ctx)
 cache_pkg_ctx *
 cache_open_pkg(cache_cat_ctx *cat_ctx, const char *name)
 {
-	return q_vdb_open_pkg(cat_ctx, name);
+	cache_pkg_ctx *ret = q_vdb_open_pkg(cat_ctx, name);
+	ret->repo = ((cache_ctx *)cat_ctx->ctx)->repo;
+	return ret;
 }
 
 cache_pkg_ctx *
@@ -154,7 +170,7 @@ cache_next_pkg(cache_cat_ctx *cat_ctx)
 				ctx->ebuilddir_pkg_ctx = NULL;
 			} else {
 				if ((p = strstr(ret->name, ".ebuild")) == NULL) {
-					q_vdb_close_pkg(ret);
+					cache_close_pkg(ret);
 					ret = NULL;
 				} else {
 					/* "zap" the pkg such that it looks like CAT/P */
@@ -475,6 +491,9 @@ cache_close_meta(cache_pkg_meta *cache)
 void
 cache_close_pkg(cache_pkg_ctx *pkg_ctx)
 {
+	/* avoid free of cache_ctx' repo by q_vdb_close_pkg */
+	if (((cache_ctx *)pkg_ctx->cat_ctx->ctx)->repo == pkg_ctx->repo)
+		pkg_ctx->repo = NULL;
 	q_vdb_close_pkg(pkg_ctx);
 }
 

diff --git a/libq/cache.h b/libq/cache.h
index a4c5803..0157824 100644
--- a/libq/cache.h
+++ b/libq/cache.h
@@ -24,6 +24,7 @@ typedef struct cache_ctx {
 	q_vdb_pkg_ctx *ebuilddir_pkg_ctx;
 	q_vdb_cat_ctx *ebuilddir_cat_ctx;
 	q_vdb_ctx ebuilddir_ctx;
+	char *repo;
 } cache_ctx;
 #define cache_cat_ctx q_vdb_cat_ctx
 #define cache_pkg_ctx q_vdb_pkg_ctx

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     95c95c9840129ec9d32be0bddda924f4426481b0
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Apr 23 08:41:21 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue Apr 23 08:41:21 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=95c95c98

q_vdb_open2: introduce version that is optionally quiet

For probing, it may be nice not to emit a warning for each probe.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/vdb.c | 14 +++++++++++---
 libq/vdb.h |  1 +
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/libq/vdb.c b/libq/vdb.c
index 6b973e8..a32ba53 100644
--- a/libq/vdb.c
+++ b/libq/vdb.c
@@ -18,13 +18,14 @@
 #include <xalloc.h>
 
 q_vdb_ctx *
-q_vdb_open(const char *sroot, const char *svdb)
+q_vdb_open2(const char *sroot, const char *svdb, bool quiet)
 {
 	q_vdb_ctx *ctx = xmalloc(sizeof(*ctx));
 
 	ctx->portroot_fd = open(sroot, O_RDONLY|O_CLOEXEC|O_PATH);
 	if (ctx->portroot_fd == -1) {
-		warnp("could not open root: %s", sroot);
+		if (!quiet)
+			warnp("could not open root: %s", sroot);
 		goto f_error;
 	}
 
@@ -35,7 +36,8 @@ q_vdb_open(const char *sroot, const char *svdb)
 	/* Cannot use O_PATH as we want to use fdopendir() */
 	ctx->vdb_fd = openat(ctx->portroot_fd, svdb, O_RDONLY|O_CLOEXEC);
 	if (ctx->vdb_fd == -1) {
-		warnp("could not open vdb: %s (in root %s)", svdb, sroot);
+		if (!quiet)
+			warnp("could not open vdb: %s (in root %s)", svdb, sroot);
 		goto cp_error;
 	}
 
@@ -54,6 +56,12 @@ q_vdb_open(const char *sroot, const char *svdb)
 	return NULL;
 }
 
+q_vdb_ctx *
+q_vdb_open(const char *sroot, const char *svdb)
+{
+	return q_vdb_open2(sroot, svdb, false);
+}
+
 void
 q_vdb_close(q_vdb_ctx *ctx)
 {

diff --git a/libq/vdb.h b/libq/vdb.h
index 80c318c..102f5a9 100644
--- a/libq/vdb.h
+++ b/libq/vdb.h
@@ -38,6 +38,7 @@ typedef int (q_vdb_pkg_cb)(q_vdb_pkg_ctx *, void *priv);
 typedef int (q_vdb_cat_filter)(q_vdb_cat_ctx *, void *priv);
 
 q_vdb_ctx *q_vdb_open(const char *sroot, const char *svdb);
+q_vdb_ctx *q_vdb_open2(const char *sroot, const char *svdb, bool quiet);
 void q_vdb_close(q_vdb_ctx *ctx);
 int q_vdb_filter_cat(const struct dirent *de);
 q_vdb_cat_ctx *q_vdb_open_cat(q_vdb_ctx *ctx, const char *name);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     7d91bbe43a2c53b1431a8529848532deebae5524
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Apr 25 07:42:28 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Apr 25 07:42:28 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=7d91bbe4

libq/cache: add mode for plain ebuilds (not using any cache)

Make cache a slight misnomer by also reading ebuilds directly if no
cache appears available.  This way cache can be used to transparently
query a repository in the best possible way.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 175 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 libq/cache.h |  10 +++-
 2 files changed, 171 insertions(+), 14 deletions(-)

diff --git a/libq/cache.c b/libq/cache.c
index 31178e7..eec39db 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -12,6 +12,7 @@
 #include <fcntl.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+#include <ctype.h>
 #include <xalloc.h>
 
 #include "cache.h"
@@ -20,7 +21,7 @@
 
 #ifdef EBUG
 static void
-cache_dump(portage_cache *cache)
+cache_dump(cache_pkg_meta *cache)
 {
 	if (!cache)
 		errf("Cache is empty !");
@@ -41,11 +42,6 @@ cache_dump(portage_cache *cache)
 	printf("PROVIDE    : %s\n", cache->PROVIDE);
 	printf("EAPI       : %s\n", cache->EAPI);
 	printf("PROPERTIES : %s\n", cache->PROPERTIES);
-	if (!cache->atom) return;
-	printf("CATEGORY   : %s\n", cache->atom->CATEGORY);
-	printf("PN         : %s\n", cache->atom->PN);
-	printf("PV         : %s\n", cache->atom->PV);
-	printf("PVR        : %s\n", cache->atom->PVR);
 }
 #endif
 
@@ -59,7 +55,7 @@ cache_open(const char *sroot, const char *portdir)
 	char buf[_Q_PATH_MAX];
 
 	snprintf(buf, sizeof(buf), "%s/%s", portdir, portcachedir_md5);
-	dir = q_vdb_open(sroot, buf);
+	dir = q_vdb_open2(sroot, buf, true);
 	if (dir != NULL) {
 		ret = xmalloc(sizeof(cache_ctx));
 		ret->dir_ctx = dir;
@@ -68,7 +64,7 @@ cache_open(const char *sroot, const char *portdir)
 	}
 
 	snprintf(buf, sizeof(buf), "%s/%s", portdir, portcachedir_pms);
-	dir = q_vdb_open(sroot, buf);
+	dir = q_vdb_open2(sroot, buf, true);
 	if (dir != NULL) {
 		ret = xmalloc(sizeof(cache_ctx));
 		ret->dir_ctx = dir;
@@ -76,6 +72,17 @@ cache_open(const char *sroot, const char *portdir)
 		return ret;
 	}
 
+	dir = q_vdb_open2(sroot, portdir, true);
+	if (dir != NULL) {
+		ret = xmalloc(sizeof(cache_ctx));
+		ret->dir_ctx = dir;
+		ret->cachetype = CACHE_EBUILD;
+		return ret;
+	}
+
+	warnf("could not open repository at %s (under root %s)",
+			portdir, sroot);
+
 	return NULL;
 }
 
@@ -98,7 +105,10 @@ cache_open_cat(cache_ctx *ctx, const char *name)
 cache_cat_ctx *
 cache_next_cat(cache_ctx *ctx)
 {
-	return q_vdb_next_cat(ctx->dir_ctx);
+	cache_cat_ctx *ret = q_vdb_next_cat(ctx->dir_ctx);
+	if (ret != NULL)
+		ret->ctx = (q_vdb_ctx *)ctx;
+	return ret;
 }
 
 void
@@ -116,7 +126,48 @@ cache_open_pkg(cache_cat_ctx *cat_ctx, const char *name)
 cache_pkg_ctx *
 cache_next_pkg(cache_cat_ctx *cat_ctx)
 {
-	return q_vdb_next_pkg(cat_ctx);
+	cache_ctx *ctx = (cache_ctx *)(cat_ctx->ctx);
+	cache_pkg_ctx *ret = NULL;
+
+	if (ctx->cachetype == CACHE_EBUILD) {
+		char *p;
+
+		/* serve *.ebuild files each as separate pkg_ctx with name set
+		 * to CAT/P like in VDB and metadata */
+		do {
+			if (ctx->ebuilddir_pkg_ctx == NULL) {
+				q_vdb_ctx *pkgdir = &ctx->ebuilddir_ctx;
+
+				if ((ctx->ebuilddir_pkg_ctx = q_vdb_next_pkg(cat_ctx)) == NULL)
+					return NULL;
+
+				pkgdir->portroot_fd = -1;
+				pkgdir->vdb_fd = cat_ctx->fd;
+				pkgdir->dir = NULL;
+
+				ctx->ebuilddir_cat_ctx =
+					q_vdb_open_cat(pkgdir, ctx->ebuilddir_pkg_ctx->name);
+			}
+
+			ret = q_vdb_next_pkg(ctx->ebuilddir_cat_ctx);
+			if (ret == NULL) {
+				ctx->ebuilddir_pkg_ctx = NULL;
+			} else {
+				if ((p = strstr(ret->name, ".ebuild")) == NULL) {
+					q_vdb_close_pkg(ret);
+					ret = NULL;
+				} else {
+					/* "zap" the pkg such that it looks like CAT/P */
+					ret->cat_ctx = cat_ctx;
+					*p = '\0';
+				}
+			}
+		} while (ret == NULL);
+	} else {
+		ret = q_vdb_next_pkg(cat_ctx);
+	}
+
+	return ret;
 }
 
 static cache_pkg_meta *
@@ -292,6 +343,103 @@ err:
 	return NULL;
 }
 
+static cache_pkg_meta *
+cache_read_file_ebuild(cache_pkg_ctx *pkg_ctx)
+{
+	FILE *f;
+	struct stat s;
+	cache_pkg_meta *ret = NULL;
+	size_t len;
+	char *p;
+	char *q;
+	char *r;
+	char **key;
+
+	if ((f = fdopen(pkg_ctx->fd, "r")) == NULL)
+		goto err;
+
+	if (fstat(pkg_ctx->fd, &s) != 0)
+		goto err;
+
+	len = sizeof(*ret) + s.st_size + 1;
+	ret = xzalloc(len);
+	p = (char *)ret;
+	ret->_data = p + sizeof(*ret);
+	if ((off_t)fread(ret->_data, 1, s.st_size, f) != s.st_size)
+		goto err;
+
+	p = ret->_data;
+	do {
+		if ((p = strchr(p, '\n')) == NULL)
+			break;
+		q = p;
+		while (*p >= 'A' && *p <= 'Z')
+			p++;
+
+		key = NULL;
+		if (q < p && *p == '=') {
+			*p++ = '\0';
+			/* match variable against which ones we look for */
+#define match_key(X) else if (strcmp(q, #X) == 0) key = &ret->X
+			if (1 == 0); /* dummy for syntax */
+			match_key(DEPEND);
+			match_key(RDEPEND);
+			match_key(SLOT);
+			match_key(SRC_URI);
+			match_key(RESTRICT);
+			match_key(HOMEPAGE);
+			match_key(LICENSE);
+			match_key(DESCRIPTION);
+			match_key(KEYWORDS);
+			match_key(IUSE);
+			match_key(CDEPEND);
+			match_key(PDEPEND);
+			match_key(EAPI);
+			match_key(REQUIRED_USE);
+#undef match_key
+		}
+
+		if (key != NULL) {
+			q = p;
+			if (*q == '"' || *q == '\'') {
+				/* find matching quote */
+				q = ++p;
+				do {
+					while (*p != '\0' && *p != *q)
+						p++;
+					if (*p == *q) {
+						for (r = p - 1; r > q; r--)
+							if (*r != '\\')
+								break;
+						if (r != q && (p - 1 - r) % 2 == 1)
+							continue;
+					}
+					break;
+				} while (1);
+			} else {
+				/* find first whitespace */
+				while (!isspace((int)*p))
+					p++;
+			}
+			*p = '\0';
+			*key = q;
+		}
+	} while (p != NULL);
+
+	fclose(f);
+	pkg_ctx->fd = -1;
+
+	return ret;
+
+err:
+	if (f)
+		fclose(f);
+	pkg_ctx->fd = -1;
+	if (ret)
+		cache_close_meta(ret);
+	return NULL;
+}
+
 cache_pkg_meta *
 cache_pkg_read(cache_pkg_ctx *pkg_ctx)
 {
@@ -308,6 +456,8 @@ cache_pkg_read(cache_pkg_ctx *pkg_ctx)
 		return cache_read_file_md5(pkg_ctx);
 	} else if (ctx->cachetype == CACHE_METADATA_PMS) {
 		return cache_read_file_pms(pkg_ctx);
+	} else if (ctx->cachetype == CACHE_EBUILD) {
+		return cache_read_file_ebuild(pkg_ctx);
 	}
 
 	warn("Unknown metadata cache type!");
@@ -351,7 +501,6 @@ cache_foreach_pkg(const char *sroot, const char *portdir,
 		}
 	}
 
-	cache_close(ctx);
 	return ret;
 }
 
@@ -382,14 +531,14 @@ cache_foreach_pkg_sorted(const char *sroot, const char *portdir,
 
 	cat_cnt = scandirat(ctx->dir_ctx->vdb_fd,
 			".", &cat_de, q_vdb_filter_cat, catsortfunc);
-	for (c = 0; c < cat_cnt; ++c) {
+	for (c = 0; c < cat_cnt; c++) {
 		cat_ctx = cache_open_cat(ctx, cat_de[c]->d_name);
 		if (!cat_ctx)
 			continue;
 
 		pkg_cnt = scandirat(ctx->dir_ctx->vdb_fd,
 				cat_de[c]->d_name, &pkg_de, q_vdb_filter_pkg, pkgsortfunc);
-		for (p = 0; p < pkg_cnt; ++p) {
+		for (p = 0; p < pkg_cnt; p++) {
 			if (pkg_de[p]->d_name[0] == '-')
 				continue;
 

diff --git a/libq/cache.h b/libq/cache.h
index c16efac..a4c5803 100644
--- a/libq/cache.h
+++ b/libq/cache.h
@@ -15,7 +15,15 @@
 
 typedef struct cache_ctx {
 	q_vdb_ctx *dir_ctx;
-	enum { CACHE_UNSET = 0, CACHE_METADATA_MD5, CACHE_METADATA_PMS } cachetype;
+	enum {
+		CACHE_UNSET = 0,
+		CACHE_METADATA_MD5,
+		CACHE_METADATA_PMS,
+		CACHE_EBUILD,
+	} cachetype;
+	q_vdb_pkg_ctx *ebuilddir_pkg_ctx;
+	q_vdb_cat_ctx *ebuilddir_cat_ctx;
+	q_vdb_ctx ebuilddir_ctx;
 } cache_ctx;
 #define cache_cat_ctx q_vdb_cat_ctx
 #define cache_pkg_ctx q_vdb_pkg_ctx

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     04213a5dfc76db2caa821d41d1cb471fa3556707
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sat Apr 20 16:03:13 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sat Apr 20 16:03:13 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=04213a5d

libq/cache: rework to be vdb-like enumerator abstraction

Using this code, one can walk through a repository's metadata cache
(md5-cache or (PMS) cache) and read cache entries conveniently.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/cache.c | 250 ++++++++++++++++++++++++++++++++++++++++++++++++++---------
 libq/cache.h |  37 ++++++---
 2 files changed, 240 insertions(+), 47 deletions(-)

diff --git a/libq/cache.c b/libq/cache.c
index a130381..31178e7 100644
--- a/libq/cache.c
+++ b/libq/cache.c
@@ -14,8 +14,9 @@
 #include <sys/stat.h>
 #include <xalloc.h>
 
-#include "atom.h"
 #include "cache.h"
+#include "scandirat.h"
+#include "vdb.h"
 
 #ifdef EBUG
 static void
@@ -48,34 +49,92 @@ cache_dump(portage_cache *cache)
 }
 #endif
 
-static portage_cache *cache_read_file_pms(const char *file);
-static portage_cache *cache_read_file_md5(const char *file);
-
-portage_cache *
-cache_read_file(int portcachedir_type, const char *file)
+static const char portcachedir_pms[] = "metadata/cache";
+static const char portcachedir_md5[] = "metadata/md5-cache";
+cache_ctx *
+cache_open(const char *sroot, const char *portdir)
 {
-	if (portcachedir_type == CACHE_METADATA_MD5)
-		return(cache_read_file_md5(file));
-	else if (portcachedir_type == CACHE_METADATA_PMS)
-		return(cache_read_file_pms(file));
-	warn("Unknown metadata cache type!");
+	q_vdb_ctx *dir;
+	cache_ctx *ret;
+	char buf[_Q_PATH_MAX];
+
+	snprintf(buf, sizeof(buf), "%s/%s", portdir, portcachedir_md5);
+	dir = q_vdb_open(sroot, buf);
+	if (dir != NULL) {
+		ret = xmalloc(sizeof(cache_ctx));
+		ret->dir_ctx = dir;
+		ret->cachetype = CACHE_METADATA_MD5;
+		return ret;
+	}
+
+	snprintf(buf, sizeof(buf), "%s/%s", portdir, portcachedir_pms);
+	dir = q_vdb_open(sroot, buf);
+	if (dir != NULL) {
+		ret = xmalloc(sizeof(cache_ctx));
+		ret->dir_ctx = dir;
+		ret->cachetype = CACHE_METADATA_PMS;
+		return ret;
+	}
+
 	return NULL;
 }
 
-static portage_cache *
-cache_read_file_pms(const char *file)
+void
+cache_close(cache_ctx *ctx)
+{
+	q_vdb_close(ctx->dir_ctx);
+	free(ctx);
+}
+
+cache_cat_ctx *
+cache_open_cat(cache_ctx *ctx, const char *name)
+{
+	cache_cat_ctx *ret = q_vdb_open_cat(ctx->dir_ctx, name);
+	if (ret != NULL)
+		ret->ctx = (q_vdb_ctx *)ctx;
+	return ret;
+}
+
+cache_cat_ctx *
+cache_next_cat(cache_ctx *ctx)
+{
+	return q_vdb_next_cat(ctx->dir_ctx);
+}
+
+void
+cache_close_cat(cache_cat_ctx *cat_ctx)
+{
+	q_vdb_close_cat(cat_ctx);
+}
+
+cache_pkg_ctx *
+cache_open_pkg(cache_cat_ctx *cat_ctx, const char *name)
+{
+	return q_vdb_open_pkg(cat_ctx, name);
+}
+
+cache_pkg_ctx *
+cache_next_pkg(cache_cat_ctx *cat_ctx)
+{
+	return q_vdb_next_pkg(cat_ctx);
+}
+
+static cache_pkg_meta *
+cache_read_file_pms(cache_pkg_ctx *pkg_ctx)
 {
 	struct stat s;
 	char *ptr;
 	FILE *f;
-	portage_cache *ret = NULL;
+	cache_pkg_meta *ret = NULL;
 	size_t len;
+	char buf[_Q_PATH_MAX];
 
-	if ((f = fopen(file, "r")) == NULL)
+	if ((f = fdopen(pkg_ctx->fd, "r")) == NULL)
 		goto err;
 
-	if (fstat(fileno(f), &s) != 0)
+	if (fstat(pkg_ctx->fd, &s) != 0)
 		goto err;
+
 	len = sizeof(*ret) + s.st_size + 1;
 	ret = xzalloc(len);
 	ptr = (char*)ret;
@@ -83,11 +142,10 @@ cache_read_file_pms(const char *file)
 	if ((off_t)fread(ret->_data, 1, s.st_size, f) != s.st_size)
 		goto err;
 
-	ret->atom = atom_explode(file);
 	ret->DEPEND = ret->_data;
 #define next_line(curr, next) \
 	if ((ptr = strchr(ret->curr, '\n')) == NULL) { \
-		warn("Invalid cache file '%s'", file); \
+		warn("Invalid cache file for '%s'", buf); \
 		goto err; \
 	} \
 	ret->next = ptr+1; \
@@ -110,35 +168,42 @@ cache_read_file_pms(const char *file)
 #undef next_line
 	ptr = strchr(ptr+1, '\n');
 	if (ptr == NULL) {
-		warn("Invalid cache file '%s' - could not find end of cache data", file);
+		warn("Invalid cache file for '%s' - could not find end of cache data",
+				buf);
 		goto err;
 	}
 	*ptr = '\0';
 
 	fclose(f);
+	pkg_ctx->fd = -1;
 
 	return ret;
 
 err:
-	if (f) fclose(f);
-	if (ret) cache_free(ret);
+	if (f)
+		fclose(f);
+	pkg_ctx->fd = -1;
+	if (ret)
+		cache_close_meta(ret);
 	return NULL;
 }
 
-static portage_cache *
-cache_read_file_md5(const char *file)
+static cache_pkg_meta *
+cache_read_file_md5(cache_pkg_ctx *pkg_ctx)
 {
 	struct stat s;
 	char *ptr, *endptr;
 	FILE *f;
-	portage_cache *ret = NULL;
+	cache_pkg_meta *ret = NULL;
 	size_t len;
+	char buf[_Q_PATH_MAX];
 
-	if ((f = fopen(file, "r")) == NULL)
+	if ((f = fdopen(pkg_ctx->fd, "r")) == NULL)
 		goto err;
 
-	if (fstat(fileno(f), &s) != 0)
+	if (fstat(pkg_ctx->fd, &s) != 0)
 		goto err;
+
 	len = sizeof(*ret) + s.st_size + 1;
 	ret = xzalloc(len);
 	ptr = (char*)ret;
@@ -146,8 +211,6 @@ cache_read_file_md5(const char *file)
 	if ((off_t)fread(ret->_data, 1, s.st_size, f) != s.st_size)
 		goto err;
 
-	ret->atom = atom_explode(file);
-
 	/* We have a block of key=value\n data.
 	 * KEY=VALUE\n
 	 * Where KEY does NOT contain:
@@ -166,7 +229,8 @@ cache_read_file_md5(const char *file)
 	ptr = ret->_data;
 	endptr = strchr(ptr, '\0');
 	if (endptr == NULL) {
-			warn("Invalid cache file '%s' - could not find end of cache data", file);
+			warn("Invalid cache file for '%s': "
+					"could not find end of cache data", buf);
 			goto err;
 	}
 
@@ -176,14 +240,14 @@ cache_read_file_md5(const char *file)
 		keyptr = ptr;
 		valptr = strchr(ptr, '=');
 		if (valptr == NULL) {
-			warn("Invalid cache file '%s' val", file);
+			warn("Invalid cache file for '%s': missing val", buf);
 			goto err;
 		}
 		*valptr = '\0';
 		valptr++;
 		ptr = strchr(valptr, '\n');
 		if (ptr == NULL) {
-			warn("Invalid cache file '%s' key", file);
+			warn("Invalid cache file for '%s': missing key", buf);
 			goto err;
 		}
 		*ptr = '\0';
@@ -209,26 +273,140 @@ cache_read_file_md5(const char *file)
 		assign_var(REQUIRED_USE);
 		assign_var(_eclasses_);
 		assign_var(_md5_);
-		warn("Cache file '%s' with unknown key %s", file, keyptr);
+		warn("Cache file for '%s' has unknown key %s", buf, keyptr);
 	}
 #undef assign_var
 #undef assign_var_cmp
 
 	fclose(f);
+	pkg_ctx->fd = -1;
 
 	return ret;
 
 err:
-	if (f) fclose(f);
-	if (ret) cache_free(ret);
+	if (f)
+		fclose(f);
+	pkg_ctx->fd = -1;
+	if (ret)
+		cache_close_meta(ret);
+	return NULL;
+}
+
+cache_pkg_meta *
+cache_pkg_read(cache_pkg_ctx *pkg_ctx)
+{
+	cache_ctx *ctx = (cache_ctx *)(pkg_ctx->cat_ctx->ctx);
+
+	if (pkg_ctx->fd == -1) {
+		pkg_ctx->fd = openat(pkg_ctx->cat_ctx->fd, pkg_ctx->name,
+				O_RDONLY|O_CLOEXEC);
+		if (pkg_ctx->fd == -1)
+			return NULL;
+	}
+
+	if (ctx->cachetype == CACHE_METADATA_MD5) {
+		return cache_read_file_md5(pkg_ctx);
+	} else if (ctx->cachetype == CACHE_METADATA_PMS) {
+		return cache_read_file_pms(pkg_ctx);
+	}
+
+	warn("Unknown metadata cache type!");
 	return NULL;
 }
 
 void
-cache_free(portage_cache *cache)
+cache_close_meta(cache_pkg_meta *cache)
 {
 	if (!cache)
 		errf("Cache is empty !");
-	atom_implode(cache->atom);
 	free(cache);
 }
+
+void
+cache_close_pkg(cache_pkg_ctx *pkg_ctx)
+{
+	q_vdb_close_pkg(pkg_ctx);
+}
+
+int
+cache_foreach_pkg(const char *sroot, const char *portdir,
+		q_vdb_pkg_cb callback, void *priv, q_vdb_cat_filter filter)
+{
+	cache_ctx *ctx;
+	cache_cat_ctx *cat_ctx;
+	cache_pkg_ctx *pkg_ctx;
+	int ret;
+
+	ctx = cache_open(sroot, portdir);
+	if (!ctx)
+		return EXIT_FAILURE;
+
+	ret = 0;
+	while ((cat_ctx = cache_next_cat(ctx))) {
+		if (filter && !filter(cat_ctx, priv))
+			continue;
+		while ((pkg_ctx = cache_next_pkg(cat_ctx))) {
+			ret |= callback(pkg_ctx, priv);
+			cache_close_pkg(pkg_ctx);
+		}
+	}
+
+	cache_close(ctx);
+	return ret;
+}
+
+int
+cache_foreach_pkg_sorted(const char *sroot, const char *portdir,
+		q_vdb_pkg_cb callback, void *priv,
+		void *catsortfunc, void *pkgsortfunc)
+{
+	cache_ctx *ctx;
+	cache_cat_ctx *cat_ctx;
+	cache_pkg_ctx *pkg_ctx;
+	int ret = 0;
+	int c;
+	int p;
+	int cat_cnt;
+	int pkg_cnt;
+	struct dirent **cat_de;
+	struct dirent **pkg_de;
+
+	ctx = cache_open(sroot, portdir);
+	if (!ctx)
+		return EXIT_FAILURE;
+
+	if (catsortfunc == NULL)
+		catsortfunc = alphasort;
+	if (pkgsortfunc == NULL)
+		pkgsortfunc = alphasort;
+
+	cat_cnt = scandirat(ctx->dir_ctx->vdb_fd,
+			".", &cat_de, q_vdb_filter_cat, catsortfunc);
+	for (c = 0; c < cat_cnt; ++c) {
+		cat_ctx = cache_open_cat(ctx, cat_de[c]->d_name);
+		if (!cat_ctx)
+			continue;
+
+		pkg_cnt = scandirat(ctx->dir_ctx->vdb_fd,
+				cat_de[c]->d_name, &pkg_de, q_vdb_filter_pkg, pkgsortfunc);
+		for (p = 0; p < pkg_cnt; ++p) {
+			if (pkg_de[p]->d_name[0] == '-')
+				continue;
+
+			pkg_ctx = cache_open_pkg(cat_ctx, pkg_de[p]->d_name);
+			if (!pkg_ctx)
+				continue;
+
+			ret |= callback(pkg_ctx, priv);
+
+			cache_close_pkg(pkg_ctx);
+		}
+		scandir_free(pkg_de, pkg_cnt);
+
+		cache_close_cat(cat_ctx);
+	}
+	scandir_free(cat_de, cat_cnt);
+
+	cache_close(ctx);
+	return ret;
+}

diff --git a/libq/cache.h b/libq/cache.h
index 93421df..c16efac 100644
--- a/libq/cache.h
+++ b/libq/cache.h
@@ -11,6 +11,14 @@
 #define _CACHE_H 1
 
 #include "atom.h"
+#include "vdb.h"
+
+typedef struct cache_ctx {
+	q_vdb_ctx *dir_ctx;
+	enum { CACHE_UNSET = 0, CACHE_METADATA_MD5, CACHE_METADATA_PMS } cachetype;
+} cache_ctx;
+#define cache_cat_ctx q_vdb_cat_ctx
+#define cache_pkg_ctx q_vdb_pkg_ctx
 
 typedef struct {
 	char *_data;
@@ -30,23 +38,30 @@ typedef struct {
 	char *PROVIDE;       /* line 14 */
 	char *EAPI;
 	char *PROPERTIES;
-	depend_atom *atom;
 	/* These are MD5-Cache only */
 	char *DEFINED_PHASES;
 	char *REQUIRED_USE;
 	char *_eclasses_;
 	char *_md5_;
-} portage_cache;
+} cache_pkg_meta;
 
-portage_cache *
-cache_read_file(int portcachedir_type, const char *file);
-void cache_free(portage_cache *cache);
+typedef int (cache_pkg_cb)(cache_pkg_ctx *, void *priv);
+typedef int (cache_cat_filter)(cache_cat_ctx *, void *priv);
 
-enum {
-	CACHE_EBUILD = 1,
-	CACHE_METADATA = 2,
-	CACHE_METADATA_PMS = 10,
-	CACHE_METADATA_MD5 = 11,
-};
+cache_ctx *cache_open(const char *sroot, const char *portdir);
+void cache_close(cache_ctx *ctx);
+cache_cat_ctx *cache_open_cat(cache_ctx *ctx, const char *name);
+cache_cat_ctx *cache_next_cat(cache_ctx *ctx);
+void cache_close_cat(cache_cat_ctx *cat_ctx);
+cache_pkg_ctx *cache_open_pkg(cache_cat_ctx *cat_ctx, const char *name);
+cache_pkg_ctx *cache_next_pkg(cache_cat_ctx *cat_ctx);
+cache_pkg_meta *cache_pkg_read(cache_pkg_ctx *pkg_ctx);
+void cache_close_meta(cache_pkg_meta *cache);
+void cache_close_pkg(cache_pkg_ctx *pkg_ctx);
+int cache_foreach_pkg(const char *sroot, const char *portdir,
+		cache_pkg_cb callback, void *priv, cache_cat_filter filter);
+int cache_foreach_pkg_sorted(const char *sroot, const char *portdir,
+		cache_pkg_cb callback, void *priv,
+		void *catsortfunc, void *pkgsortfunc);
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     868b2b4b4f5140da7188389acf9f719fcf343ab2
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Apr 19 11:47:29 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Apr 19 11:47:29 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=868b2b4b

atom: allow category-only atoms

This allows to match categories via atom matches, using e.g. mail-client/

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/libq/atom.c b/libq/atom.c
index 4c06c1a..58565bc 100644
--- a/libq/atom.c
+++ b/libq/atom.c
@@ -160,6 +160,11 @@ atom_explode(const char *atom)
 	if ((ptr = strrchr(ret->CATEGORY, '/')) != NULL) {
 		ret->PN = ptr + 1;
 		*ptr = '\0';
+
+		/* set PN to NULL if there's nothing */
+		if (ret->PN[0] == '\0')
+			ret->PN = NULL;
+
 		/* eat extra crap in case it exists, this is a feature to allow
 		 * /path/to/pkg.ebuild, doesn't work with prefix operators
 		 * though */
@@ -170,6 +175,13 @@ atom_explode(const char *atom)
 		ret->CATEGORY = NULL;
 	}
 
+	if (ret->PN == NULL) {
+		/* atom has no name, this is it */
+		ret->P = NULL;
+		ret->PVR = NULL;
+		return ret;
+	}
+
 	/* hunt down build with USE dependencies */
 	if ((ptr = strrchr(ret->PN, ']')) != NULL && ptr[1] == '\0' &&
 			(ptr = strrchr(ret->PN, '[')) != NULL)
@@ -378,15 +390,22 @@ int
 atom_compare(const depend_atom *a1, const depend_atom *a2)
 {
 	/* sanity check that at most one has operators */
-	if (a1->pfx_op != ATOM_OP_NONE || a1->sfx_op != ATOM_OP_NONE) {
-		/* this is bogus, so punt it */
-		if (a2->pfx_op != ATOM_OP_NONE || a2->sfx_op != ATOM_OP_NONE)
+	if (a1->pfx_op != ATOM_OP_NONE ||
+			a1->sfx_op != ATOM_OP_NONE ||
+			a1->blocker != ATOM_BL_NONE)
+	{
+		/* is the other also having operators, then punt it */
+		if (a2->pfx_op != ATOM_OP_NONE ||
+				a2->sfx_op != ATOM_OP_NONE ||
+				a2->blocker != ATOM_BL_NONE)
 			return NOT_EQUAL;
+
 		/* swap a1 & a2 so that a2 is the atom with operators */
 		const depend_atom *as = a2;
 		a2 = a1;
 		a1 = as;
 	}
+
 	atom_operator pfx_op = a2->pfx_op;
 	atom_operator sfx_op = a2->sfx_op;
 
@@ -437,6 +456,9 @@ atom_compare(const depend_atom *a1, const depend_atom *a2)
 	if (a1->PN && a2->PN) {
 		if (strcmp(a1->PN, a2->PN))
 			return NOT_EQUAL;
+	} else if (a1->CATEGORY && a2->CATEGORY) {
+		/* if CAT is set, and one side has empty PN, accept as match */
+		return a2->blocker != ATOM_BL_NONE ? NOT_EQUAL : EQUAL;
 	} else if (a1->PN || a2->PN)
 		return NOT_EQUAL;
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     83f3df9d2df983c10b370be423606be53536a909
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Mar 25 20:44:01 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Mar 25 20:44:01 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=83f3df9d

libq: split out cache-related funcs from main/qsearch

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/Makefile.am |   1 +
 libq/Makefile.in |  24 ++++--
 libq/cache.c     | 234 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 libq/cache.h     |  50 ++++++++++++
 4 files changed, 302 insertions(+), 7 deletions(-)

diff --git a/libq/Makefile.am b/libq/Makefile.am
index 80b651c..26647ee 100644
--- a/libq/Makefile.am
+++ b/libq/Makefile.am
@@ -3,6 +3,7 @@ QFILES = \
 	atom.c atom.h \
 	basename.c basename.h \
 	busybox.h \
+	cache.c cache.h \
 	colors.c colors.h \
 	copy_file.c copy_file.h \
 	eat_file.c eat_file.h \

diff --git a/libq/Makefile.in b/libq/Makefile.in
index a5e43fc..6c529c2 100644
--- a/libq/Makefile.in
+++ b/libq/Makefile.in
@@ -241,13 +241,14 @@ CONFIG_CLEAN_FILES =
 CONFIG_CLEAN_VPATH_FILES =
 LTLIBRARIES = $(noinst_LTLIBRARIES)
 libq_la_LIBADD =
-am__objects_1 = libq_la-atom.lo libq_la-basename.lo libq_la-colors.lo \
-	libq_la-copy_file.lo libq_la-eat_file.lo libq_la-hash_fd.lo \
-	libq_la-human_readable.lo libq_la-md5_sha1_sum.lo \
-	libq_la-prelink.lo libq_la-profile.lo libq_la-rmspace.lo \
-	libq_la-safe_io.lo libq_la-scandirat.lo libq_la-set.lo \
-	libq_la-vdb.lo libq_la-xarray.lo libq_la-xchdir.lo \
-	libq_la-xmkdir.lo libq_la-xregex.lo libq_la-xsystem.lo
+am__objects_1 = libq_la-atom.lo libq_la-basename.lo libq_la-cache.lo \
+	libq_la-colors.lo libq_la-copy_file.lo libq_la-eat_file.lo \
+	libq_la-hash_fd.lo libq_la-human_readable.lo \
+	libq_la-md5_sha1_sum.lo libq_la-prelink.lo libq_la-profile.lo \
+	libq_la-rmspace.lo libq_la-safe_io.lo libq_la-scandirat.lo \
+	libq_la-set.lo libq_la-vdb.lo libq_la-xarray.lo \
+	libq_la-xchdir.lo libq_la-xmkdir.lo libq_la-xregex.lo \
+	libq_la-xsystem.lo
 am_libq_la_OBJECTS = $(am__objects_1)
 libq_la_OBJECTS = $(am_libq_la_OBJECTS)
 AM_V_lt = $(am__v_lt_@AM_V@)
@@ -1444,6 +1445,7 @@ QFILES = \
 	atom.c atom.h \
 	basename.c basename.h \
 	busybox.h \
+	cache.c cache.h \
 	colors.c colors.h \
 	copy_file.c copy_file.h \
 	eat_file.c eat_file.h \
@@ -1528,6 +1530,7 @@ distclean-compile:
 
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-atom.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-basename.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-cache.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-colors.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-copy_file.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libq_la-eat_file.Plo@am__quote@
@@ -1582,6 +1585,13 @@ libq_la-basename.lo: basename.c
 @AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
 @am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libq_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libq_la-basename.lo `test -f 'basename.c' || echo '$(srcdir)/'`basename.c
 
+libq_la-cache.lo: cache.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libq_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libq_la-cache.lo -MD -MP -MF $(DEPDIR)/libq_la-cache.Tpo -c -o libq_la-cache.lo `test -f 'cache.c' || echo '$(srcdir)/'`cache.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libq_la-cache.Tpo $(DEPDIR)/libq_la-cache.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='cache.c' object='libq_la-cache.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libq_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libq_la-cache.lo `test -f 'cache.c' || echo '$(srcdir)/'`cache.c
+
 libq_la-colors.lo: colors.c
 @am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libq_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libq_la-colors.lo -MD -MP -MF $(DEPDIR)/libq_la-colors.Tpo -c -o libq_la-colors.lo `test -f 'colors.c' || echo '$(srcdir)/'`colors.c
 @am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libq_la-colors.Tpo $(DEPDIR)/libq_la-colors.Plo

diff --git a/libq/cache.c b/libq/cache.c
new file mode 100644
index 0000000..a130381
--- /dev/null
+++ b/libq/cache.c
@@ -0,0 +1,234 @@
+/*
+ * Copyright 2005-2019 Gentoo Foundation
+ * Distributed under the terms of the GNU General Public License v2
+ *
+ * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
+ * Copyright 2005-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2018-     Fabian Groffen  - <grobian@gentoo.org>
+ */
+
+#include "main.h"
+
+#include <fcntl.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <xalloc.h>
+
+#include "atom.h"
+#include "cache.h"
+
+#ifdef EBUG
+static void
+cache_dump(portage_cache *cache)
+{
+	if (!cache)
+		errf("Cache is empty !");
+
+	printf("DEPEND     : %s\n", cache->DEPEND);
+	printf("RDEPEND    : %s\n", cache->RDEPEND);
+	printf("SLOT       : %s\n", cache->SLOT);
+	printf("SRC_URI    : %s\n", cache->SRC_URI);
+	printf("RESTRICT   : %s\n", cache->RESTRICT);
+	printf("HOMEPAGE   : %s\n", cache->HOMEPAGE);
+	printf("LICENSE    : %s\n", cache->LICENSE);
+	printf("DESCRIPTION: %s\n", cache->DESCRIPTION);
+	printf("KEYWORDS   : %s\n", cache->KEYWORDS);
+	printf("INHERITED  : %s\n", cache->INHERITED);
+	printf("IUSE       : %s\n", cache->IUSE);
+	printf("CDEPEND    : %s\n", cache->CDEPEND);
+	printf("PDEPEND    : %s\n", cache->PDEPEND);
+	printf("PROVIDE    : %s\n", cache->PROVIDE);
+	printf("EAPI       : %s\n", cache->EAPI);
+	printf("PROPERTIES : %s\n", cache->PROPERTIES);
+	if (!cache->atom) return;
+	printf("CATEGORY   : %s\n", cache->atom->CATEGORY);
+	printf("PN         : %s\n", cache->atom->PN);
+	printf("PV         : %s\n", cache->atom->PV);
+	printf("PVR        : %s\n", cache->atom->PVR);
+}
+#endif
+
+static portage_cache *cache_read_file_pms(const char *file);
+static portage_cache *cache_read_file_md5(const char *file);
+
+portage_cache *
+cache_read_file(int portcachedir_type, const char *file)
+{
+	if (portcachedir_type == CACHE_METADATA_MD5)
+		return(cache_read_file_md5(file));
+	else if (portcachedir_type == CACHE_METADATA_PMS)
+		return(cache_read_file_pms(file));
+	warn("Unknown metadata cache type!");
+	return NULL;
+}
+
+static portage_cache *
+cache_read_file_pms(const char *file)
+{
+	struct stat s;
+	char *ptr;
+	FILE *f;
+	portage_cache *ret = NULL;
+	size_t len;
+
+	if ((f = fopen(file, "r")) == NULL)
+		goto err;
+
+	if (fstat(fileno(f), &s) != 0)
+		goto err;
+	len = sizeof(*ret) + s.st_size + 1;
+	ret = xzalloc(len);
+	ptr = (char*)ret;
+	ret->_data = ptr + sizeof(*ret);
+	if ((off_t)fread(ret->_data, 1, s.st_size, f) != s.st_size)
+		goto err;
+
+	ret->atom = atom_explode(file);
+	ret->DEPEND = ret->_data;
+#define next_line(curr, next) \
+	if ((ptr = strchr(ret->curr, '\n')) == NULL) { \
+		warn("Invalid cache file '%s'", file); \
+		goto err; \
+	} \
+	ret->next = ptr+1; \
+	*ptr = '\0';
+	next_line(DEPEND, RDEPEND)
+	next_line(RDEPEND, SLOT)
+	next_line(SLOT, SRC_URI)
+	next_line(SRC_URI, RESTRICT)
+	next_line(RESTRICT, HOMEPAGE)
+	next_line(HOMEPAGE, LICENSE)
+	next_line(LICENSE, DESCRIPTION)
+	next_line(DESCRIPTION, KEYWORDS)
+	next_line(KEYWORDS, INHERITED)
+	next_line(INHERITED, IUSE)
+	next_line(IUSE, CDEPEND)
+	next_line(CDEPEND, PDEPEND)
+	next_line(PDEPEND, PROVIDE)
+	next_line(PROVIDE, EAPI)
+	next_line(EAPI, PROPERTIES)
+#undef next_line
+	ptr = strchr(ptr+1, '\n');
+	if (ptr == NULL) {
+		warn("Invalid cache file '%s' - could not find end of cache data", file);
+		goto err;
+	}
+	*ptr = '\0';
+
+	fclose(f);
+
+	return ret;
+
+err:
+	if (f) fclose(f);
+	if (ret) cache_free(ret);
+	return NULL;
+}
+
+static portage_cache *
+cache_read_file_md5(const char *file)
+{
+	struct stat s;
+	char *ptr, *endptr;
+	FILE *f;
+	portage_cache *ret = NULL;
+	size_t len;
+
+	if ((f = fopen(file, "r")) == NULL)
+		goto err;
+
+	if (fstat(fileno(f), &s) != 0)
+		goto err;
+	len = sizeof(*ret) + s.st_size + 1;
+	ret = xzalloc(len);
+	ptr = (char*)ret;
+	ret->_data = ptr + sizeof(*ret);
+	if ((off_t)fread(ret->_data, 1, s.st_size, f) != s.st_size)
+		goto err;
+
+	ret->atom = atom_explode(file);
+
+	/* We have a block of key=value\n data.
+	 * KEY=VALUE\n
+	 * Where KEY does NOT contain:
+	 * \0 \n =
+	 * And VALUE does NOT contain:
+	 * \0 \n
+	 * */
+#define assign_var_cmp(keyname, cmpkey) \
+	if (strncmp(keyptr, cmpkey, strlen(cmpkey)) == 0) { \
+		ret->keyname = valptr; \
+		continue; \
+	}
+#define assign_var(keyname) \
+	assign_var_cmp(keyname, #keyname);
+
+	ptr = ret->_data;
+	endptr = strchr(ptr, '\0');
+	if (endptr == NULL) {
+			warn("Invalid cache file '%s' - could not find end of cache data", file);
+			goto err;
+	}
+
+	while (ptr != NULL && ptr != endptr) {
+		char *keyptr;
+		char *valptr;
+		keyptr = ptr;
+		valptr = strchr(ptr, '=');
+		if (valptr == NULL) {
+			warn("Invalid cache file '%s' val", file);
+			goto err;
+		}
+		*valptr = '\0';
+		valptr++;
+		ptr = strchr(valptr, '\n');
+		if (ptr == NULL) {
+			warn("Invalid cache file '%s' key", file);
+			goto err;
+		}
+		*ptr = '\0';
+		ptr++;
+
+		assign_var(CDEPEND);
+		assign_var(DEPEND);
+		assign_var(DESCRIPTION);
+		assign_var(EAPI);
+		assign_var(HOMEPAGE);
+		assign_var(INHERITED);
+		assign_var(IUSE);
+		assign_var(KEYWORDS);
+		assign_var(LICENSE);
+		assign_var(PDEPEND);
+		assign_var(PROPERTIES);
+		assign_var(PROVIDE);
+		assign_var(RDEPEND);
+		assign_var(RESTRICT);
+		assign_var(SLOT);
+		assign_var(SRC_URI);
+		assign_var(DEFINED_PHASES);
+		assign_var(REQUIRED_USE);
+		assign_var(_eclasses_);
+		assign_var(_md5_);
+		warn("Cache file '%s' with unknown key %s", file, keyptr);
+	}
+#undef assign_var
+#undef assign_var_cmp
+
+	fclose(f);
+
+	return ret;
+
+err:
+	if (f) fclose(f);
+	if (ret) cache_free(ret);
+	return NULL;
+}
+
+void
+cache_free(portage_cache *cache)
+{
+	if (!cache)
+		errf("Cache is empty !");
+	atom_implode(cache->atom);
+	free(cache);
+}

diff --git a/libq/cache.h b/libq/cache.h
new file mode 100644
index 0000000..971417c
--- /dev/null
+++ b/libq/cache.h
@@ -0,0 +1,50 @@
+/*
+ * Copyright 2005-2019 Gentoo Foundation
+ * Distributed under the terms of the GNU General Public License v2
+ *
+ * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
+ * Copyright 2005-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2019-     Fabian Groffen  - <grobian@gentoo.org>
+ */
+
+#ifndef _CACHE_H
+#define _CACHE_H 1
+
+typedef struct {
+	char *_data;
+	char *DEPEND;        /* line 1 */
+	char *RDEPEND;
+	char *SLOT;
+	char *SRC_URI;
+	char *RESTRICT;      /* line 5 */
+	char *HOMEPAGE;
+	char *LICENSE;
+	char *DESCRIPTION;
+	char *KEYWORDS;
+	char *INHERITED;     /* line 10 */
+	char *IUSE;
+	char *CDEPEND;
+	char *PDEPEND;
+	char *PROVIDE;       /* line 14 */
+	char *EAPI;
+	char *PROPERTIES;
+	depend_atom *atom;
+	/* These are MD5-Cache only */
+	char *DEFINED_PHASES;
+	char *REQUIRED_USE;
+	char *_eclasses_;
+	char *_md5_;
+} portage_cache;
+
+portage_cache *
+cache_read_file(int portcachedir_type, const char *file);
+void cache_free(portage_cache *cache);
+
+enum {
+	CACHE_EBUILD = 1,
+	CACHE_METADATA = 2,
+	CACHE_METADATA_PMS = 10,
+	CACHE_METADATA_MD5 = 11,
+};
+
+#endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     268c11a1f565820c6f4612cb21770249bf7d41db
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Mar 11 20:16:51 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Mar 11 20:16:51 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=268c11a1

set: fix signedness warnings

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/set.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libq/set.c b/libq/set.c
index 68799fe..f48921a 100644
--- a/libq/set.c
+++ b/libq/set.c
@@ -77,7 +77,7 @@ static set *
 add_set_unique(const char *name, set *q, bool *unique)
 {
 	char *mname = xstrdup(name);
-	int hash;
+	unsigned int hash;
 	int pos;
 	elem *ll;
 	elem *w;
@@ -123,7 +123,7 @@ static bool
 contains_set(char *s, set *q)
 {
 	char *mname = xstrdup(s);
-	int hash;
+	unsigned int hash;
 	int pos;
 	elem *w;
 	bool found;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     53b93e0ee76ff5de732ffd54d56e24a0ef1b71ba
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Mar  7 21:52:16 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Mar  7 21:52:16 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=53b93e0e

rmspace: add copyright

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/rmspace.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/libq/rmspace.c b/libq/rmspace.c
index b8ac5a3..40c3808 100644
--- a/libq/rmspace.c
+++ b/libq/rmspace.c
@@ -1,5 +1,14 @@
+/*
+ * Copyright 2005-2019 Gentoo Foundation
+ * Distributed under the terms of the GNU General Public License v2
+ *
+ * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
+ * Copyright 2005-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2019-     Fabian Groffen  - <grobian@gentoo.org>
+ */
 
-/* removed leading/trailing extraneous white space */
+
+/* remove leading/trailing extraneous white space */
 static char *rmspace_len(char *s, size_t len)
 {
 	char *p;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     62fe544cf61aa9ae3e3560323d84f221c6c1a375
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Wed Feb 27 20:41:01 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Feb 27 20:50:16 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=62fe544c

xarray: add xarrayget function to retrieve a given item

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xarray.c | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/libq/xarray.c b/libq/xarray.c
index 3ed9872..0ab1c5a 100644
--- a/libq/xarray.c
+++ b/libq/xarray.c
@@ -18,15 +18,24 @@ typedef struct {
  * already do not permit pushing of NULL pointers), but we can't put it in the
  * increment phase as that will cause a load beyond the bounds of valid memory.
  */
+/* TODO: remove ele = NULL after checking all consumers don't rely on this */
 #define array_for_each(arr, n, ele) \
-	for (n = 0, ele = array_cnt(arr) ? arr->eles[n] : NULL; \
-	     n < array_cnt(arr) && (ele = arr->eles[n]); \
-	     ++n)
+	for (n = 0, ele = NULL; n < array_cnt(arr) && (ele = arr->eles[n]); n++)
+#define array_for_each_rev(arr, n, ele) \
+	for (n = array_cnt(arr); n-- > 0 && (ele = arr->eles[n]); /*nothing*/)
+#define array_get_elem(arr, n) (arr->eles[n])
 #define array_init_decl { .eles = NULL, .num = 0, }
 #define array_cnt(arr) (arr)->num
 #define DECLARE_ARRAY(arr) array_t _##arr = array_init_decl, *arr = &_##arr
 #define ARRAY_INC_SIZE 32
 
+static void *xarrayget(array_t *arr, size_t idx)
+{
+	if (idx >= arr->num)
+		return NULL;
+	return arr->eles[idx];
+}
+
 /* Push a pointer to memory we already hold and don't want to release.  Do not
  * mix xarraypush_ptr usage with the other push funcs which duplicate memory.
  * The free stage won't know which pointers to release directly.
@@ -51,7 +60,10 @@ static void *xarraypush(array_t *arr, const void *ele, size_t ele_len)
 static void xarraydelete_ptr(array_t *arr, size_t elem)
 {
 	arr->num--;
-	memmove(&arr->eles[elem], &arr->eles[elem + 1], arr->num - elem);
+	if (elem < arr->num)
+		memmove(&arr->eles[elem], &arr->eles[elem + 1],
+				sizeof(arr->eles[0]) * (arr->num - elem));
+	arr->eles[arr->num] = NULL;
 }
 
 static void xarraydelete(array_t *arr, size_t elem)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     ace156ca85c3d5c804ec06e7cb4172f5b6d279e1
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Feb 26 08:32:50 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Wed Feb 27 20:49:47 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ace156ca

xarray: improve malloc efficiency, allow removing elements

Pre-allocate members such that we don't have to re-alloc for every
insertion.
Allow removing elements from the array, such that they can be used as
containers of running lists.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xarray.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/libq/xarray.c b/libq/xarray.c
index 64334bb..3ed9872 100644
--- a/libq/xarray.c
+++ b/libq/xarray.c
@@ -1,14 +1,16 @@
 /*
- * Copyright 2003-2018 Gentoo Foundation
+ * Copyright 2003-2019 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2003-2007 Ned Ludd        - <solar@gentoo.org>
  * Copyright 2004-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2018-     Fabian Groffen  - <grobian@gentoo.org>
  */
 
 typedef struct {
 	void **eles;
 	size_t num;
+	size_t len;
 } array_t;
 
 #define xrealloc_array(ptr, size, ele_size) xrealloc(ptr, (size) * (ele_size))
@@ -23,6 +25,7 @@ typedef struct {
 #define array_init_decl { .eles = NULL, .num = 0, }
 #define array_cnt(arr) (arr)->num
 #define DECLARE_ARRAY(arr) array_t _##arr = array_init_decl, *arr = &_##arr
+#define ARRAY_INC_SIZE 32
 
 /* Push a pointer to memory we already hold and don't want to release.  Do not
  * mix xarraypush_ptr usage with the other push funcs which duplicate memory.
@@ -31,7 +34,10 @@ typedef struct {
 static void *xarraypush_ptr(array_t *arr, void *ele)
 {
 	size_t n = arr->num++;
-	arr->eles = xrealloc_array(arr->eles, arr->num, sizeof(ele));
+	if (arr->num > arr->len) {
+		arr->len += ARRAY_INC_SIZE;
+		arr->eles = xrealloc_array(arr->eles, arr->len, sizeof(ele));
+	}
 	arr->eles[n] = ele;
 	return ele;
 }
@@ -42,6 +48,18 @@ static void *xarraypush(array_t *arr, const void *ele, size_t ele_len)
 #define xarraypush_str(arr, ele) xarraypush(arr, ele, strlen(ele) + 1 /*NUL*/)
 #define xarraypush_struct(arr, ele) xarraypush(arr, ele, sizeof(*(ele)))
 
+static void xarraydelete_ptr(array_t *arr, size_t elem)
+{
+	arr->num--;
+	memmove(&arr->eles[elem], &arr->eles[elem + 1], arr->num - elem);
+}
+
+static void xarraydelete(array_t *arr, size_t elem)
+{
+	free(arr->eles[elem]);
+	xarraydelete_ptr(arr, elem);
+}
+
 /* Useful for people who call xarraypush_ptr as it does not free any of the
  * pointers in the eles list.
  */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e2319b0dcf7b54d71ddfd1e3eca712c49e83d6fe
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Feb  5 14:19:22 2019 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue Feb  5 14:19:22 2019 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e2319b0d

atom_explode: ignore slot matching operators

Bug: https://bugs.gentoo.org/673750
Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/atom_explode.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libq/atom_explode.c b/libq/atom_explode.c
index a4ba569..de42103 100644
--- a/libq/atom_explode.c
+++ b/libq/atom_explode.c
@@ -1,9 +1,10 @@
 /*
- * Copyright 2005-2018 Gentoo Foundation
+ * Copyright 2005-2019 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
  *
  * Copyright 2005-2008 Ned Ludd        - <solar@gentoo.org>
  * Copyright 2005-2014 Mike Frysinger  - <vapier@gentoo.org>
+ * Copyright 2018-     Fabian Groffen  - <grobian@gentoo.org>
  */
 
 typedef enum { VER_ALPHA=0, VER_BETA, VER_PRE, VER_RC, VER_NORM, VER_P } atom_suffixes;
@@ -161,6 +162,10 @@ atom_explode(const char *atom)
 	if ((ptr = strrchr(ret->CATEGORY, ':')) != NULL) {
 		ret->SLOT = ptr + 1;
 		*ptr = '\0';
+
+		/* ignore slots that are about package matching */
+		if (ret->SLOT[0] == '=' || ret->SLOT[0] == '*')
+			ret->SLOT = NULL;
 	}
 
 	/* see if we have any suffix operators */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     5539ab9cf34f303b7e11c5989d8cde8f1ed57043
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Dec 20 19:58:18 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Dec 20 19:58:18 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=5539ab9c

rm_rf_at: ensure return code makes sense

Don't blindly ignore errors, just run statements that should succeed and
register success.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xmkdir.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/libq/xmkdir.c b/libq/xmkdir.c
index aa93905..bc89fe0 100644
--- a/libq/xmkdir.c
+++ b/libq/xmkdir.c
@@ -49,6 +49,7 @@ rm_rf_at(int dfd, const char *path)
 	int subdfd;
 	DIR *dir;
 	struct dirent *de;
+	int ret = 0;
 
 	/* Cannot use O_PATH as we want to use fdopendir() */
 	subdfd = openat(dfd, path, O_RDONLY|O_CLOEXEC|O_NOFOLLOW);
@@ -73,17 +74,16 @@ rm_rf_at(int dfd, const char *path)
 						!(st.st_mode & S_IFDIR))
 					errp("could not unlink %s", de->d_name);
 			}
-			rm_rf_at(subdfd, de->d_name);
-			unlinkat(subdfd, de->d_name, AT_REMOVEDIR);
+			ret |= rm_rf_at(subdfd, de->d_name);
 		}
 	}
 
-	unlinkat(dfd, path, AT_REMOVEDIR);
+	ret |= unlinkat(dfd, path, AT_REMOVEDIR);
 
 	/* this also does close(subdfd); */
 	closedir(dir);
 
-	return 0;
+	return ret;
 }
 
 static int

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     fb74fe22e3eb430ff28bbd884d2ef1546f891051
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Dec 20 19:44:35 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Dec 20 19:44:35 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=fb74fe22

rm_rf: make shallow wrapper around rm_rf_at

Since rm_rf_at now removes the object being pointed at, rm_rf doesn't
have to try doing that again.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xmkdir.c | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)

diff --git a/libq/xmkdir.c b/libq/xmkdir.c
index ed30d6d..aa93905 100644
--- a/libq/xmkdir.c
+++ b/libq/xmkdir.c
@@ -89,18 +89,7 @@ rm_rf_at(int dfd, const char *path)
 static int
 rm_rf(const char *path)
 {
-	rm_rf_at(AT_FDCWD, path);
-
-	if (rmdir(path) == 0)
-		return 0;
-
-	/* if path is a symlink, unlink it */
-	if (unlink(path) == 0)
-		return 0;
-
-	/* XXX: we don't handle:
-	 *      trailing slashes: `rm -rf a/b/c/` -> need to change to a/b/c */
-	return -1;
+	return rm_rf_at(AT_FDCWD, path);
 }
 
 static int

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     5d274dbc605c72b43c13f8ac61c988eae40531ce
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Dec 20 11:44:41 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Dec 20 18:22:36 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=5d274dbc

rm_rf_at: remove the object path points to

To better emulate 'rm -Rf' behaviour, if path points to a file, just
remove it.  If it points to a directory, remove that directory instead
of leaving the (now) empty directory behind.

Signed-off-by: Fabian Groffen <grobian <AT> gentoo.org>

 libq/xmkdir.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/libq/xmkdir.c b/libq/xmkdir.c
index 9a120ae..ed30d6d 100644
--- a/libq/xmkdir.c
+++ b/libq/xmkdir.c
@@ -58,7 +58,7 @@ rm_rf_at(int dfd, const char *path)
 	dir = fdopendir(subdfd);
 	if (!dir) {
 		close(subdfd);
-		return -1;
+		return unlinkat(dfd, path, 0);
 	}
 
 	while ((de = readdir(dir)) != NULL) {
@@ -78,6 +78,8 @@ rm_rf_at(int dfd, const char *path)
 		}
 	}
 
+	unlinkat(dfd, path, AT_REMOVEDIR);
+
 	/* this also does close(subdfd); */
 	closedir(dir);
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     6d6a52d90e5a16f86947fd163aac23d3b7b66f32
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Apr  5 17:02:56 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Apr  5 17:02:56 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=6d6a52d9

xstrdup: avoid warning about clobbering t

When using optimisation, the compiler does something to a char pointer
it doesn't do to a void pointer, so use a couple of casts to avoid a
clobber warning.

 libq/xstrdup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libq/xstrdup.c b/libq/xstrdup.c
index 6924d21..e069c9d 100644
--- a/libq/xstrdup.c
+++ b/libq/xstrdup.c
@@ -29,16 +29,16 @@
 
 static char *xstrdup(const char *s)
 {
-	char *t;
+	void *t;
 
 	if (s == NULL)
 		return NULL;
 
-	t = strdup(s);
+	t = (void *)strdup(s);
 	if (unlikely(t == NULL))
 		err("Out of memory");
 
-	return t;
+	return (char *)t;
 }
 
 static char *xstrdup_len(const char *s, size_t *len)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     ccc0b73bade57d56a6512f7f174aa04c1018be5b
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Apr  5 13:29:10 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Apr  5 13:29:10 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ccc0b73b

atom_explode: get version letters comparing properly again

 libq/atom_explode.c | 35 +++++++++++++++++++----------------
 1 file changed, 19 insertions(+), 16 deletions(-)

diff --git a/libq/atom_explode.c b/libq/atom_explode.c
index faf24ff..99a60d0 100644
--- a/libq/atom_explode.c
+++ b/libq/atom_explode.c
@@ -256,26 +256,29 @@ atom_explode(const char *atom)
 		ret->suffixes[idx] = t;
 	}
 
-	/* allow for 1 optional suffix letter, must be following a number */
-	ptr = ret->PV + strlen(ret->PV);
-	if (ptr[-1] >= 'a' && ptr[-1] <= 'z' &&
-			ptr - 2 > ret->PV && ptr[-2] >= '0' && ptr[-2] <= '9')
-	{
-		ret->letter = ptr[-1];
-		--ptr;
-	}
+	/* skip back to the "end" */
+	for (ptr = ret->PV; *ptr != '\0' && *ptr != '_'; ptr++)
+		;
+	ptr--;
 
-	/* eat the trailing version number [-.0-9]+ */
-	while (--ptr > ret->PV) {
-		if (*ptr == '-') {
-			*ptr = '\0';
-			break;
-		} else if (*ptr != '.' && !isdigit(*ptr))
+	/* allow for 1 optional suffix letter */
+	if (*ptr >= 'a' && *ptr <= 'z')
+		ret->letter = *ptr--;
+
+	/* eat the trailing version number [.0-9]+ */
+	while (ptr > ret->PV) {
+		if (*ptr != '.' && !isdigit(*ptr))
 			break;
+		ptr--;
 	}
 
-	ptr = stpcpy(ret->PVR, ret->PV);
-	sprintf(ptr, "-r%i", ret->PR_int);
+	if (ptr != ret->PV) {
+		/* PV isn't exactly a number */
+		ret->PV = ret->PVR = NULL;
+	} else {
+		ptr = stpcpy(ret->PVR, ret->PV);
+		sprintf(ptr, "-r%i", ret->PR_int);
+	}
 
 	return ret;
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     85f7604d713d1c7fce3ea31798faee4487a90dc7
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Apr  5 12:43:59 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Apr  5 12:43:59 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=85f7604d

atom_explode: properly set fields for no-version case

atom_explode/basic test wasn't run due to no dependencies in Makefile,
hence I didn't see this before

 libq/atom_explode.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libq/atom_explode.c b/libq/atom_explode.c
index e52f149..faf24ff 100644
--- a/libq/atom_explode.c
+++ b/libq/atom_explode.c
@@ -200,6 +200,8 @@ atom_explode(const char *atom)
 
 	if (ptr == NULL) {
 		/* atom has no version, this is it */
+		strcpy(ret->P, ret->PN);
+		ret->PVR = NULL;
 		return ret;
 	}
 	ret->PV = ptr;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     af8611e773e9c2ec7c475787f262a33ceec3e98e
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Tue Apr  3 17:20:01 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Tue Apr  3 19:57:51 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=af8611e7

scandirat: deal properly with flexible struct member

 libq/scandirat.c | 31 +++++++++++++++++++------------
 1 file changed, 19 insertions(+), 12 deletions(-)

diff --git a/libq/scandirat.c b/libq/scandirat.c
index f0b3201..ac45223 100644
--- a/libq/scandirat.c
+++ b/libq/scandirat.c
@@ -14,20 +14,17 @@
 
 #if !defined(HAVE_SCANDIRAT)
 
-#if defined(_DIRENT_HAVE_D_RECLEN)
-# define reclen(de) ((de)->d_reclen)
-#else
-# define reclen(de) (sizeof(*(de)) + strlen((de)->d_name))
-#endif
-
 static int
 scandirat(int dir_fd, const char *dir, struct dirent ***dirlist,
 	int (*filter)(const struct dirent *),
 	int (*compar)(const struct dirent **, const struct dirent **))
 {
-	int fd, cnt;
+	int fd;
 	DIR *dirp;
 	struct dirent *de, **ret;
+	size_t retlen = 0;
+	size_t retsize = 0;
+#define INCRSZ 64
 
 	/* Cannot use O_PATH as we want to use fdopendir() */
 	fd = openat(dir_fd, dir, O_RDONLY|O_CLOEXEC);
@@ -40,22 +37,32 @@ scandirat(int dir_fd, const char *dir, struct dirent ***dirlist,
 	}
 
 	ret = NULL;
-	cnt = 0;
 	while ((de = readdir(dirp))) {
+		size_t sdesz;
+		size_t sdenamelen;
+
 		if (filter(de) == 0)
 			continue;
 
-		ret = realloc(ret, sizeof(*ret) * (cnt + 1));
-		ret[cnt++] = xmemdup(de, reclen(de));
+		if (retlen == retsize) {
+			retsize += INCRSZ;
+			ret = xrealloc(ret, sizeof(*ret) * retsize);
+		}
+		sdesz = (void *)de->d_name - (void *)de;
+		sdenamelen = strlen(de->d_name) + 1;
+		ret[retlen] = xmalloc(sdesz + sdenamelen);
+		memcpy(ret[retlen], de, sdesz);
+		strncpy(ret[retlen]->d_name, de->d_name, sdenamelen);
+		retlen++;
 	}
 	*dirlist = ret;
 
-	qsort(ret, cnt, sizeof(*ret), (void *)compar);
+	qsort(ret, retlen, sizeof(*ret), (void *)compar);
 
 	/* closes underlying fd */
 	closedir(dirp);
 
-	return cnt;
+	return (int)retlen;
 }
 
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     09dad6bdeb1e750ce0bbb25c6123f3ad23ca3d3a
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Mon Mar 26 18:40:43 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Mon Mar 26 18:40:43 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=09dad6bd

colors: add support for color=color mappings

This patch is based on the work by Pavel Balaev <pascal <AT> unixdev.ru>.

Add support for color=color mappings.  While at it, make color remapping
independent of the order in which they are defined in color.map.

Bug: https://bugs.gentoo.org/651546

 libq/colors.c | 65 ++++++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 44 insertions(+), 21 deletions(-)

diff --git a/libq/colors.c b/libq/colors.c
index 33f6d89..5f252b6 100644
--- a/libq/colors.c
+++ b/libq/colors.c
@@ -21,29 +21,34 @@ static const char *WHITE = _MAKE_COLOR("01", "38");
 static const char *COLOR_MAP = CONFIG_EPREFIX "etc/portage/color.map";
 
 #define COLOR _MAKE_COLOR
+#define CPAIR_VALUE_LEN 16
 
 typedef struct {
 	const char *name;
-	char value[16];
+	char value[CPAIR_VALUE_LEN];
+	char origval[CPAIR_VALUE_LEN];
 } cpairtype;
 
+#define X2(X) X, X
 static cpairtype color_pairs[] = {
-	{"blue",      COLOR("34", "01") },
-	{"brown",     COLOR("00", "33") },
-	{"darkblue",  COLOR("00", "34") },
-	{"darkgreen", COLOR("00", "32") },
-	{"darkred",   COLOR("00", "31") },
-	{"faint",     COLOR("00", "02") },
-	{"fuchsia",   COLOR("35", "01") },
-	{"green",     COLOR("32", "01") },
-	{"purple",    COLOR("00", "35") },
-	{"red",       COLOR("31", "01") },
-	{"teal",      COLOR("00", "36") },
-	{"turquoise", COLOR("36", "01") },
-	{"yellow",    COLOR("01", "33") },
-	{"white",     COLOR("01", "38") },
-	{"eol",       COLOR("00", "00") },
+	{"blue",      X2(COLOR("34", "01")) },
+	{"brown",     X2(COLOR("00", "33")) },
+	{"darkblue",  X2(COLOR("00", "34")) },
+	{"darkgreen", X2(COLOR("00", "32")) },
+	{"darkred",   X2(COLOR("00", "31")) },
+	{"faint",     X2(COLOR("00", "02")) },
+	{"fuchsia",   X2(COLOR("35", "01")) },
+	{"green",     X2(COLOR("32", "01")) },
+	{"purple",    X2(COLOR("00", "35")) },
+	{"red",       X2(COLOR("31", "01")) },
+	{"teal",      X2(COLOR("00", "36")) },
+	{"turquoise", X2(COLOR("36", "01")) },
+	{"yellow",    X2(COLOR("01", "33")) },
+	{"white",     X2(COLOR("01", "38")) },
+	{"lightgray", X2(COLOR("00", "37")) },
+	{"eol",       X2(COLOR("00", "00")) },
 };
+#undef X2
 
 static void
 color_remap(void)
@@ -73,13 +78,31 @@ color_remap(void)
 			continue;
 
 		*p++ = 0; /* split the pair */
-		for (i = 0; i < ARRAY_SIZE(color_pairs); ++i)
+		for (i = 0; i < ARRAY_SIZE(color_pairs); ++i) {
 			if (strcmp(buf, color_pairs[i].name) == 0) {
-				if (strncmp(p, "0x", 2) == 0)
-					warn("[%s=%s] RGB values in color map are not supported on line %d of %s", buf, p, lineno, COLOR_MAP);
-				else
-					snprintf(color_pairs[i].value, sizeof(color_pairs[i].value), "\e[%s", p);
+				if (strncmp(p, "0x", 2) == 0) {
+					warn("[%s=%s] RGB values in color map are not "
+							"supported on line %d of %s",
+							buf, p, lineno, COLOR_MAP);
+				} else {
+					/* color=color format support */
+					size_t n;
+					int found = 0;
+					for (n = 0; n < ARRAY_SIZE(color_pairs); n++) {
+						if (strcmp(color_pairs[n].name, p) == 0) {
+							strncpy(color_pairs[i].value,
+									color_pairs[n].origval, CPAIR_VALUE_LEN);
+							found = 1;
+							break;
+						}
+					}
+
+					if (!found)
+						snprintf(color_pairs[i].value,
+								sizeof(color_pairs[i].origval), "\e[%s", p);
+				}
 			}
+		}
 	}
 
 	free(buf);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     905449f8011e71f0cc018c9aa755cdbfb0307461
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Mar 25 14:12:33 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Mar 25 14:12:33 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=905449f8

atom_explode: fix inversion of < and <=

 libq/atom_explode.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libq/atom_explode.c b/libq/atom_explode.c
index d5d0840..1b81909 100644
--- a/libq/atom_explode.c
+++ b/libq/atom_explode.c
@@ -135,9 +135,9 @@ atom_explode(const char *atom)
 			++atom;
 			if (atom[0] == '=') {
 				++atom;
-				ret->pfx_op = ATOM_OP_NEWER_EQUAL;
-			} else
 				ret->pfx_op = ATOM_OP_NEWER;
+			} else
+				ret->pfx_op = ATOM_OP_NEWER_EQUAL;
 			break;
 		default:
 			ret->pfx_op = ATOM_OP_BLOCK;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     c9b624a2a93d75654269e9ad39a8c09450cd2fb1
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Mar 25 13:28:32 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Sun Mar 25 13:28:32 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=c9b624a2

atom_explode: support inversed ranges, e.g. !>P-x.y

For bug #608960 we need to be able to handle odd atoms like
!<logrotate-2.3.0 (as used by sys-apps/portage).  This adds support for
this construct.

 libq/atom_explode.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/libq/atom_explode.c b/libq/atom_explode.c
index 057bfa1..d5d0840 100644
--- a/libq/atom_explode.c
+++ b/libq/atom_explode.c
@@ -118,11 +118,31 @@ atom_explode(const char *atom)
 		break;
 	case '!':
 		++atom;
-		if (atom[0] == '!') {
+		switch (atom[0]) {
+		case '!':
 			++atom;
 			ret->pfx_op = ATOM_OP_BLOCK_HARD;
-		} else
+			break;
+		case '>':
+			++atom;
+			if (atom[0] == '=') {
+				++atom;
+				ret->pfx_op = ATOM_OP_OLDER;
+			} else
+				ret->pfx_op = ATOM_OP_OLDER_EQUAL;
+			break;
+		case '<':
+			++atom;
+			if (atom[0] == '=') {
+				++atom;
+				ret->pfx_op = ATOM_OP_NEWER_EQUAL;
+			} else
+				ret->pfx_op = ATOM_OP_NEWER;
+			break;
+		default:
 			ret->pfx_op = ATOM_OP_BLOCK;
+			break;
+		}
 		break;
 	}
 	strcpy(ret->CATEGORY, atom);

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     56f9a5d07c39ce39d1e66af44bdcb370071e48de
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Mar 23 16:25:59 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Mar 23 16:25:59 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=56f9a5d0

hash_fd: induce byteorder on Solaris too

 libq/hash_fd.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/libq/hash_fd.c b/libq/hash_fd.c
index 49e3af5..a88d756 100644
--- a/libq/hash_fd.c
+++ b/libq/hash_fd.c
@@ -31,7 +31,17 @@
 #ifdef HAVE_SYS_PARAM_H
 # include <sys/param.h>
 # ifndef __BYTE_ORDER
-#  define __BYTE_ORDER BYTE_ORDER
+#  ifdef BYTE_ORDER
+#   define __BYTE_ORDER BYTE_ORDER
+#  elif defined(_LITTLE_ENDIAN)
+#   define __LITTLE_ENDIAN 1234
+#   define __BIG_ENDIAN 4321
+#   define __BYTE_ORDER __LITTLE_ENDIAN
+#  elif defined(_BIG_ENDIAN)
+#   define __LITTLE_ENDIAN 1234
+#   define __BIG_ENDIAN 4321
+#   define __BYTE_ORDER __BIG_ENDIAN
+#  endif
 # endif
 # ifndef __BIG_ENDIAN
 #  define __BIG_ENDIAN BIG_ENDIAN

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     6d14e87ab7ce23453621643b562e4ef51c08ca95
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Mar 23 11:31:42 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Mar 23 11:31:42 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=6d14e87a

is_prelink_elf: squash unused variable warning

 libq/prelink.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libq/prelink.c b/libq/prelink.c
index de7b205..e8bc859 100644
--- a/libq/prelink.c
+++ b/libq/prelink.c
@@ -104,6 +104,9 @@ static bool is_prelink_elf(int fd, const char *filename)
 #else
 static bool is_prelink_elf(int fd, const char *filename)
 {
+	(void) fd;
+	(void) filename;
+
 	return false;
 }
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     5d86f949a513fb19cf3bc68c78bd3848edac0529
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Fri Mar 23 11:28:31 2018 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Mar 23 11:28:31 2018 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=5d86f949

color_remap: fix comparison of integers of different signs

Just use an int to store getline return, then ensure it is positive,
such that a cast to size_t is guaranteed to result in the same value.

 libq/colors.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/libq/colors.c b/libq/colors.c
index f90be0d..33f6d89 100644
--- a/libq/colors.c
+++ b/libq/colors.c
@@ -50,7 +50,8 @@ color_remap(void)
 {
 	FILE *fp;
 	unsigned int i;
-	size_t buflen, linelen;
+	int linelen;
+	size_t buflen;
 	char *buf;
 	char *p;
 	unsigned int lineno = 0;
@@ -59,13 +60,13 @@ color_remap(void)
 		return;
 
 	buf = NULL;
-	while ((linelen = getline(&buf, &buflen, fp)) != -1) {
+	while ((linelen = getline(&buf, &buflen, fp)) >= 0) {
 		lineno++;
 		/* eat comments */
 		if ((p = strchr(buf, '#')) != NULL)
 			*p = '\0';
 
-		rmspace_len(buf, linelen);
+		rmspace_len(buf, (size_t)linelen);
 
 		p = strchr(buf, '=');
 		if (p == NULL)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     8e9d0f3e53f2b102e47a98f419b196a2184694c2
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Dec 28 16:00:28 2017 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Dec 28 16:00:28 2017 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=8e9d0f3e

rm_rf_at: fix recurse logic for non-Linux

EISDIR is non-POSIX used by the Linux kernel, but not returned on e.g.
Darwin, so use the EISDIR shortcut, if we can, else perform a stat to
figure out if we're dealing with a directory or not.

 libq/xmkdir.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/libq/xmkdir.c b/libq/xmkdir.c
index 265cb5b..9a120ae 100644
--- a/libq/xmkdir.c
+++ b/libq/xmkdir.c
@@ -65,8 +65,14 @@ rm_rf_at(int dfd, const char *path)
 		if (!strcmp(de->d_name, ".") || !strcmp(de->d_name, ".."))
 			continue;
 		if (unlinkat(subdfd, de->d_name, 0) == -1) {
-			if (unlikely(errno != EISDIR))
-				errp("could not unlink %s", de->d_name);
+			if (unlikely(errno != EISDIR)) {
+				struct stat st;
+				/* above is a linux short-cut, we really just want to
+				 * know whether we're really with a directory or not */
+				if (fstatat(subdfd, de->d_name, &st, 0) != 0 ||
+						!(st.st_mode & S_IFDIR))
+					errp("could not unlink %s", de->d_name);
+			}
 			rm_rf_at(subdfd, de->d_name);
 			unlinkat(subdfd, de->d_name, AT_REMOVEDIR);
 		}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     913d23e85ae155edf6f9d957ecc6e938caf91683
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Dec 28 16:56:59 2017 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Fri Dec 29 11:16:09 2017 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=913d23e8

hash_fd: ensure __BYTE_ORDER is set

Fallback to using sys/param.h when __BYTE_ORDER isn't set (which it
isn't on Darwin at least)

 libq/hash_fd.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/libq/hash_fd.c b/libq/hash_fd.c
index c021267..49e3af5 100644
--- a/libq/hash_fd.c
+++ b/libq/hash_fd.c
@@ -28,6 +28,19 @@
 #include <string.h>
 #include <unistd.h>
 
+#ifdef HAVE_SYS_PARAM_H
+# include <sys/param.h>
+# ifndef __BYTE_ORDER
+#  define __BYTE_ORDER BYTE_ORDER
+# endif
+# ifndef __BIG_ENDIAN
+#  define __BIG_ENDIAN BIG_ENDIAN
+# endif
+# ifndef __LITTLE_ENDIAN
+#  define __LITTLE_ENDIAN LITTLE_ENDIAN
+# endif
+#endif
+
 #include "busybox.h"
 
 #ifdef CONFIG_SHA1SUM

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Fabian Groffen]

commit:     e5ed806b4a2784ced621efd8c5b036bddef780f8
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Thu Dec 28 16:12:54 2017 +0000
Commit:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
CommitDate: Thu Dec 28 16:12:54 2017 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=e5ed806b

atom_explode: be more careful with eating suffix letters

Make sure we don't just eat a char at the end of the atom, because if
that char appears after a -, it will be seen as version part.  An
example is xerces-c, where the parsed atom form would just be xerces.

Bug: https://bugs.gentoo.org/638816

 libq/atom_explode.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/libq/atom_explode.c b/libq/atom_explode.c
index 956ac49..057bfa1 100644
--- a/libq/atom_explode.c
+++ b/libq/atom_explode.c
@@ -218,9 +218,12 @@ atom_explode(const char *atom)
 		ret->suffixes[idx] = t;
 	}
 
-	/* allow for 1 optional suffix letter */
+	/* allow for 1 optional suffix letter, must be following a number
+	 * otherwise we eat stuff like -c, see bug #639978 */
 	ptr = ret->PN + strlen(ret->PN);
-	if (ptr[-1] >= 'a' && ptr[-1] <= 'z') {
+	if (ptr[-1] >= 'a' && ptr[-1] <= 'z' &&
+			ptr - 2 > ret->PN && ptr[-2] >= '0' && ptr[-2] <= '9')
+	{
 		ret->letter = ptr[-1];
 		--ptr;
 	}

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     2863e563f7b37d9e9cbb56551c8a4c033f802750
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Wed Dec 28 22:14:25 2016 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Wed Dec 28 22:14:25 2016 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=2863e563

copy_file: rewrite to use fds

This avoids FILE* leakage and produces a little bit smaller code.

 libq/copy_file.c | 63 ++++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 41 insertions(+), 22 deletions(-)

diff --git a/libq/copy_file.c b/libq/copy_file.c
index ee1338d..4230b73 100644
--- a/libq/copy_file.c
+++ b/libq/copy_file.c
@@ -1,31 +1,50 @@
-static int copy_file_fd(int fd_src, int fd_dst)
+static ssize_t safe_read(int fd, void *buf, size_t len)
 {
-	FILE *fp_src, *fp_dst;
-	size_t rcnt, wcnt;
-	char buf[BUFSIZE];
+	ssize_t ret;
 
-	/* dont fclose() as that implicitly close()'s */
+	while (1) {
+		ret = read(fd, buf, len);
+		if (ret >= 0)
+			break;
+		else if (errno != EINTR)
+			break;
+	}
 
-	fp_src = fdopen(fd_src, "r");
-	if (!fp_src)
-		return -1;
+	return ret;
+}
 
-	fp_dst = fdopen(fd_dst, "w");
-	if (!fp_dst)
-		return -1;
+static ssize_t safe_write(int fd, const void *buf, size_t len)
+{
+	ssize_t ret;
 
-	while (1) {
-		rcnt = fread(buf, sizeof(buf[0]), sizeof(buf), fp_src);
-		if (!rcnt) {
-			fflush(fp_dst);
-			return feof(fp_src) ? 0 : -1;
+	while (len) {
+		ret = write(fd, buf, len);
+		if (ret < 0) {
+			if (errno == EINTR)
+				continue;
+			return -1;
 		}
+		buf += ret;
+		len -= ret;
+	}
 
-		wcnt = fwrite(buf, sizeof(buf[0]), rcnt, fp_dst);
-		if (wcnt != rcnt) {
-			if (ferror(fp_dst))
-				return -1;
-			fseek(fp_src, wcnt - rcnt, SEEK_CUR);
-		}
+	return ret;
+}
+
+static int copy_file_fd(int fd_src, int fd_dst)
+{
+	ssize_t rcnt, wcnt;
+	char buf[64 * 1024];
+
+	while (1) {
+		rcnt = safe_read(fd_src, buf, sizeof(buf));
+		if (rcnt < 0)
+			return -1;
+		else if (rcnt == 0)
+			return 0;
+
+		wcnt = safe_write(fd_dst, buf, rcnt);
+		if (wcnt == -1)
+			return -1;
 	}
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     c913c735be27d2f67221abf255dc38e31a9c3bad
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Sat Nov 12 17:22:17 2016 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Sat Nov 12 17:22:17 2016 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=c913c735

human_readable: resync with upstream busybox #590932

URL: https://bugs.gentoo.org/590932
Reported-by: S. Gilles <sgilles <AT> math.umd.edu>

 libq/human_readable.c | 77 +++++++++++++++++++++++++--------------------------
 1 file changed, 37 insertions(+), 40 deletions(-)

diff --git a/libq/human_readable.c b/libq/human_readable.c
index 3a8aa34..0bb044a 100644
--- a/libq/human_readable.c
+++ b/libq/human_readable.c
@@ -13,16 +13,13 @@
  *      representations (say, powers of 1024) and manipulating coefficients.
  *      The base ten "bytes" output could be handled similarly.
  *
- *   2) This routine always outputs a decimal point and a tenths digit when
- *      display_unit != 0.  Hence, it isn't uncommon for the returned string
+ *   2) This routine outputs a decimal point and a tenths digit when
+ *      display_unit == 0.  Hence, it isn't uncommon for the returned string
  *      to have a length of 5 or 6.
  *
- *      It might be nice to add a flag to indicate no decimal digits in
- *      that case.  This could be either an additional parameter, or a
- *      special value of display_unit.  Such a flag would also be nice for du.
+ *      If block_size is also 0, no decimal digits are printed.
  *
- *      Some code to omit the decimal point and tenths digit is sketched out
- *      and "#if 0"'d below.
+ * Licensed under GPLv2, see file LICENSE in this source tree.
  */
 
 #include <stdio.h>
@@ -33,59 +30,59 @@ enum {
 	MEGABYTE = (KILOBYTE*1024),
 	GIGABYTE = (MEGABYTE*1024)
 };
-const char *make_human_readable_str(unsigned long long size,
+
+const char* make_human_readable_str(unsigned long long val,
 	unsigned long block_size, unsigned long display_unit)
 {
-	/* The code will adjust for additional (appended) units. */
-	static const char zero_and_units[] = { '0', 0, 'k', 'M', 'G', 'T' };
-	static const char fmt[] = "%'Lu";
-	static const char fmt_tenths[] = "%'Lu%s%d%c";
+	static const char unit_chars[] = {
+		'\0', 'K', 'M', 'G', 'T', 'P', 'E', 'Z', 'Y'
+	};
+
+	unsigned frac; /* 0..9 - the fractional digit */
+	const char *u;
+	const char *fmt;
 
 	static char str[21];		/* Sufficient for 64 bit unsigned integers. */
 
-	unsigned long long val;
-	int frac;
-	const char *u;
-	const char *f;
+	if (val == 0)
+		return "0";
 
-	u = zero_and_units;
-	f = fmt;
+	fmt = "%llu";
+	if (block_size > 1)
+		val *= block_size;
 	frac = 0;
-
-	val = size * block_size;
-	if (val == 0) {
-		return u;
-	}
+	u = unit_chars;
 
 	if (display_unit) {
-		val += display_unit/2;	/* Deal with rounding. */
-		val /= display_unit;	/* Don't combine with the line above!!! */
+		val += display_unit/2;  /* Deal with rounding */
+		val /= display_unit;    /* Don't combine with the line above! */
+		/* will just print it as ulonglong (below) */
 	} else {
-		++u;
-		while ((val >= KILOBYTE)
-			   && (u < zero_and_units + sizeof(zero_and_units) - 1)) {
-			f = fmt_tenths;
-			++u;
-			frac = ((((int)(val % KILOBYTE)) * 10) + (KILOBYTE/2)) / KILOBYTE;
-			val /= KILOBYTE;
+		while ((val >= 1024)
+		 /* && (u < unit_chars + sizeof(unit_chars) - 1) - always true */
+		) {
+			fmt = "%llu.%u%c";
+			u++;
+			frac = (((unsigned)val % 1024) * 10 + 1024/2) / 1024;
+			val /= 1024;
 		}
-		if (frac >= 10) {		/* We need to round up here. */
+		if (frac >= 10) { /* we need to round up here */
 			++val;
 			frac = 0;
 		}
-#if 0
-		/* Sample code to omit decimal point and tenths digit. */
-		if (1) { /* no_tenths */
-			if (frac >= 5)
+#if 1
+		/* If block_size is 0, dont print fractional part */
+		if (block_size == 0) {
+			if (frac >= 5) {
 				++val;
-			f = "%Lu%*c"; /* fmt_no_tenths */
+			}
+			fmt = "%llu%*c";
 			frac = 1;
 		}
 #endif
 	}
 
-	/* If f==fmt then 'frac' and 'u' are ignored. */
-	snprintf(str, sizeof(str), f, val, decimal_point, frac, *u);
+	snprintf(str, sizeof(str), fmt, val, frac, *u);
 
 	return str;
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     6baa58541b51e1805bc18c859a4b1d2b36b7d107
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 13 22:56:03 2016 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Sat Feb 13 22:56:03 2016 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=6baa5854

compat: drop strcasestr/asprintf fallback

We pull in strcasestr & asprintf via gnulib now, so don't need these
local copies.

 libq/compat.c | 22 ----------------------
 1 file changed, 22 deletions(-)

diff --git a/libq/compat.c b/libq/compat.c
index d5f3278..ca5bb47 100644
--- a/libq/compat.c
+++ b/libq/compat.c
@@ -8,28 +8,6 @@
 
 #define S_BLKSIZE DK_DEVID_BLKSIZE
 
-/* strcasestr is a GNU extention */
-char* strcasestr(const char *big, const char *little) {
-	char* b = alloca((strlen(big) + 1) * sizeof(char));
-	char* l = alloca((strlen(little) + 1) * sizeof(char));
-	char* off;
-	size_t i;
-	for (i = 0; big[i]; i++) b[i] = (char)tolower(big[i]);
-	for (i = 0; little[i]; i++) l[i] = (char)tolower(little[i]);
-	off = strstr(b, l);
-	return(off == NULL ? off : (char*)(big + (off - b)));
-}
-
-#undef  xasprintf
-#define xasprintf(strp, fmt, args...) \
-	do { /* xasprintf() */ \
-		char str[BUFSIZ]; \
-		if ((snprintf(str, sizeof(str)-1, fmt , ## args)) == -1) \
-			err("Out of stack space?"); \
-		str[sizeof(str)-1] = '\0'; \
-		*strp = xstrdup(str); \
-	} while (0)
-
 #elif defined(__hpux__) || defined(__MINT__)
 	/* must not include both dir.h and dirent.h on hpux11..11 & FreeMiNT */
 #elif defined(__linux__)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     f086f446267b76ee4c274bdf99a6ada9be8b73d1
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 13 23:09:18 2016 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Sat Feb 13 23:09:18 2016 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=f086f446

compat: drop redundant headers

The main porting header pulls in a bunch of these already.

 libq/compat.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/libq/compat.c b/libq/compat.c
index ca5bb47..95cb7a7 100644
--- a/libq/compat.c
+++ b/libq/compat.c
@@ -1,9 +1,6 @@
 /* Solaris compatible code */
 #ifdef __sun__
 
-#include <string.h>
-#include <ctype.h>
-#include <alloca.h>
 #include <sys/dklabel.h>
 
 #define S_BLKSIZE DK_DEVID_BLKSIZE

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     66090491b033778785f12ccd3f20cdf54e89c87a
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Thu Nov 26 08:35:45 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Thu Nov 26 08:35:45 2015 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=66090491

xarray: sync logic with latest pax-utils

This pulls in two fixes:
 - handling of empty arrays
 - invalid loads at end of arrays

URL: https://bugs.gentoo.org/553368
Reported-by: Hanno Boeck <hanno <AT> gentoo.org>

 libq/xarray.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/libq/xarray.c b/libq/xarray.c
index b4c3857..56f04da 100644
--- a/libq/xarray.c
+++ b/libq/xarray.c
@@ -12,8 +12,14 @@ typedef struct {
 } array_t;
 
 #define xrealloc_array(ptr, size, ele_size) xrealloc(ptr, (size) * (ele_size))
+/* The assignment after the check is unfortunate as we do a non-NULL check (we
+ * already do not permit pushing of NULL pointers), but we can't put it in the
+ * increment phase as that will cause a load beyond the bounds of valid memory.
+ */
 #define array_for_each(arr, n, ele) \
-	for (n = 0, ele = arr->eles[n]; n < arr->num; ++n, ele = arr->eles[n])
+	for (n = 0, ele = array_cnt(arr) ? arr->eles[n] : NULL; \
+	     n < array_cnt(arr) && (ele = arr->eles[n]); \
+	     ++n)
 #define array_init_decl { .eles = NULL, .num = 0, }
 #define array_cnt(arr) (arr)->num
 #define DECLARE_ARRAY(arr) array_t _##arr = array_init_decl, *arr = &_##arr

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     153e9ff8bec2ec4cd1e777ee4b24e477a716ee87
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Thu Oct 15 20:54:30 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Thu Oct 15 21:02:19 2015 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=153e9ff8

atom_explode: fix setup of PVR

Tests are included in the next commit.

 libq/atom_explode.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libq/atom_explode.c b/libq/atom_explode.c
index 9eb5695..2855246 100644
--- a/libq/atom_explode.c
+++ b/libq/atom_explode.c
@@ -161,7 +161,6 @@ depend_atom *atom_explode(const char *atom)
 		ret->PN = ret->CATEGORY;
 		ret->CATEGORY = NULL;
 	}
-	strcpy(ret->PVR, ret->PN);
 
 	/* find -r# */
 	ptr = ret->PN + strlen(ret->PN) - 1;
@@ -170,8 +169,7 @@ depend_atom *atom_explode(const char *atom)
 			if (ptr[0] == 'r' && ptr[-1] == '-') {
 				ret->PR_int = atoi(ptr + 1);
 				ptr[-1] = '\0';
-			} else
-				strcat(ret->PVR, "-r0");
+			}
 			break;
 		}
 		--ptr;
@@ -236,6 +234,8 @@ depend_atom *atom_explode(const char *atom)
 			break;
 	if (has_pv) {
 		ret->PV = ret->P + (ptr - ret->PN) + 1;
+		ptr = stpcpy(ret->PVR, ret->PV);
+		sprintf(ptr, "-r%i", ret->PR_int);
 	} else {
 		/* atom has no version */
 		ret->PV = ret->PVR = NULL;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     525da8b994240b63353dfc1e7fe63d67b259ef5a
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Thu Oct  8 20:37:06 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Thu Oct  8 20:37:06 2015 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=525da8b9

compat: do not include sys/dir.h on linux systems

This is an old compat header that might not exist on POSIX systems.
Since Linux C libs don't generally need it, avoid it there.

 libq/compat.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libq/compat.c b/libq/compat.c
index 8def2e9..d5f3278 100644
--- a/libq/compat.c
+++ b/libq/compat.c
@@ -32,6 +32,8 @@ char* strcasestr(const char *big, const char *little) {
 
 #elif defined(__hpux__) || defined(__MINT__)
 	/* must not include both dir.h and dirent.h on hpux11..11 & FreeMiNT */
+#elif defined(__linux__)
+	/* Linux systems do not need sys/dir.h as they are generally POSIX sane */
 #else /* __sun__ */
 # include <sys/dir.h>
 #endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     bdd7d6b82859c6cf1386619d36087e346f169795
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Sun May 31 08:28:35 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Sun May 31 08:28:38 2015 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=bdd7d6b8

xsystem: fix fchdir handling with <linux-3.5

We use O_PATH with the vdb dirs, and sometimes we try to fchdir with
those fds.  Unfortunately, fchdir doesn't work with O_PATH and linux-3.5
leading to errors like:
	>>> pkg_postrm
	merge: fchdir failed: Bad file descriptor

When fchdir fails with EBADF, assume it's because of an older kernel,
and fall back to using the /proc/self/fd/ path indirectly.

 libq/xsystem.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/libq/xsystem.c b/libq/xsystem.c
index d8a551a..71e8306 100644
--- a/libq/xsystem.c
+++ b/libq/xsystem.c
@@ -22,8 +22,16 @@ static void xsystembash(const char *command, int cwd)
 	switch (p) {
 	case 0: /* child */
 		if (cwd != AT_FDCWD)
-			if (fchdir(cwd))
-				errp("fchdir failed");
+			if (fchdir(cwd)) {
+				/* fchdir works with O_PATH starting w/linux-3.5 */
+				if (errno == EBADF) {
+					char *path;
+					xasprintf(&path, "/proc/self/fd/%i", cwd);
+					if (chdir(path))
+						errp("chdir(%s) failed", path);
+				} else
+					errp("fchdir(%i) failed", cwd);
+			}
 		execl("/bin/bash", "bash", "--norc", "--noprofile", "-c", command, NULL);
 		/* Hrm, still here ?  Maybe no bash ... */
 		_exit(execl("/bin/sh", "sh", "-c", command, NULL));

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     ee6539928328ec93f714f6426ef29edeb7f95cd4
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 15:51:28 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Tue May 19 15:51:28 2015 +0000
URL:        https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ee653992

xasprintf: support returning the len

 libq/xasprintf.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/libq/xasprintf.c b/libq/xasprintf.c
index 9d8b6d5..f2f1c58 100644
--- a/libq/xasprintf.c
+++ b/libq/xasprintf.c
@@ -23,7 +23,9 @@
 
 /* asprintf(char **strp, const char *fmt, ...); */
 #define xasprintf(strp, fmt, args...) \
-	do { \
-		if (asprintf(strp, fmt , ## args) == -1) \
+	({ \
+		int _ret = asprintf(strp, fmt , ## args); \
+		if (_ret == -1) \
 			err("Out of memory"); \
-	} while (0)
+		_ret; \
+	})

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     4992b64a6f726550801e8c56f19b98f209cc9651
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Sun Feb 22 18:05:35 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Sun Feb 22 18:05:35 2015 +0000
URL:        http://sources.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=4992b64a

atom: add a print helper for debugging

---
 libq/atom_explode.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/libq/atom_explode.c b/libq/atom_explode.c
index daa18ac..9eb5695 100644
--- a/libq/atom_explode.c
+++ b/libq/atom_explode.c
@@ -40,6 +40,20 @@ typedef struct {
 	char *P, *SLOT, *REPO;
 } depend_atom;
 
+void atom_print(const depend_atom *atom);
+void atom_print(const depend_atom *atom)
+{
+	if (atom->CATEGORY)
+		printf("%s/", atom->CATEGORY);
+	printf("%s", atom->P);
+	if (atom->PR_int)
+		printf("-r%i", atom->PR_int);
+	if (atom->SLOT)
+		printf(":%s", atom->SLOT);
+	if (atom->REPO)
+		printf("::%s", atom->REPO);
+}
+
 #ifdef _USE_CACHE
 static depend_atom *_atom_cache = NULL;
 static size_t _atom_cache_len = 0;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     2d17adcdcd2c9a16879998d3ab362b6dbfaa4150
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Sun Feb 22 08:55:58 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Sun Feb 22 08:55:58 2015 +0000
URL:        http://sources.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=2d17adcd

virtuals: delete unused code, mark things static, and constify

---
 libq/virtuals.c | 32 +++++++++++---------------------
 1 file changed, 11 insertions(+), 21 deletions(-)

diff --git a/libq/virtuals.c b/libq/virtuals.c
index 8d8da03..18663e0 100644
--- a/libq/virtuals.c
+++ b/libq/virtuals.c
@@ -20,10 +20,8 @@ struct queue_t {
 
 typedef struct queue_t queue;
 
-queue *del_set(char *s, queue *q, int *ok);
-queue *add_set(const char *vv, const char *ss, queue *q);
-
-static queue *append_set(queue *q, queue *ll)
+_q_static queue *
+append_set(queue *q, queue *ll)
 {
 	queue *z;
 
@@ -39,7 +37,8 @@ static queue *append_set(queue *q, queue *ll)
 }
 
 /* add a set to a cache */
-queue *add_set(const char *vv, const char *ss, queue *q)
+_q_static queue *
+add_set(const char *vv, const char *ss, queue *q)
 {
 	queue *ll;
 	char *s, *ptr;
@@ -83,7 +82,8 @@ queue *add_set(const char *vv, const char *ss, queue *q)
 }
 
 /* remove a set from a cache. matches ->name and frees name,item */
-queue *del_set(char *s, queue *q, int *ok)
+_q_static queue *
+del_set(char *s, queue *q, int *ok)
 {
 	queue *ll, *list, *old;
 	ll = q;
@@ -117,8 +117,8 @@ queue *del_set(char *s, queue *q, int *ok)
 }
 
 /* clear out a list */
-void free_sets(queue *list);
-void free_sets(queue *list)
+_q_static void
+free_sets(queue *list)
 {
 	queue *ll, *q;
 	ll = list;
@@ -131,20 +131,10 @@ void free_sets(queue *list)
 	}
 }
 
-char *virtual(char *name, queue *list);
-char *virtual(char *name, queue *list)
+void print_sets(const queue *list);
+void print_sets(const queue *list)
 {
-	queue *ll;
-	for (ll = list; ll != NULL; ll = ll->next)
-		if ((strcmp(ll->name, name)) == 0)
-			return ll->item;
-	return NULL;
-}
-
-void print_sets(queue *list);
-void print_sets(queue *list)
-{
-	queue *ll;
+	const queue *ll;
 	for (ll = list; ll != NULL; ll = ll->next)
 		printf("%s -> %s\n", ll->name, ll->item);
 }

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     296dea574aff001ae7baa86be0ec902609558867
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Sun Feb 22 08:56:26 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Sun Feb 22 08:56:26 2015 +0000
URL:        http://sources.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=296dea57

mkdir: add a mkdir_p_at helper

---
 libq/xmkdir.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/libq/xmkdir.c b/libq/xmkdir.c
index 600e6d0..cc428aa 100644
--- a/libq/xmkdir.c
+++ b/libq/xmkdir.c
@@ -1,11 +1,11 @@
 /* Emulate `mkdir -p -m MODE PATH` */
-static int mkdir_p(const char *path, mode_t mode)
+static int mkdir_p_at(int dfd, const char *path, mode_t mode)
 {
 	char *_p, *p, *s;
 
 	/* Assume that most of the time, only the last element
 	 * is missing.  So if we can mkdir it right away, bail. */
-	if (mkdir(path, mode) == 0 || errno == EEXIST)
+	if (mkdirat(dfd, path, mode) == 0 || errno == EEXIST)
 		return 0;
 
 	/* Build up the whole tree */
@@ -19,13 +19,13 @@ static int mkdir_p(const char *path, mode_t mode)
 		/* Find the next path element */
 		s = strchr(p, '/');
 		if (!s) {
-			mkdir(_p, mode);
+			mkdirat(dfd, _p, mode);
 			break;
 		}
 
 		/* Make it */
 		*s = '\0';
-		mkdir(_p, mode);
+		mkdirat(dfd, _p, mode);
 		*s = '/';
 
 		p = s;
@@ -35,6 +35,10 @@ static int mkdir_p(const char *path, mode_t mode)
 
 	return 0;
 }
+static int mkdir_p(const char *path, mode_t mode)
+{
+	return mkdir_p_at(AT_FDCWD, path, mode);
+}
 
 /* Emulate `rm -rf PATH` */
 _q_static int rm_rf_at(int dfd, const char *path)

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     de40ef688f60ccfaffaaf17bd250825d6372c9bc
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 21 18:00:18 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Sat Feb 21 18:00:18 2015 +0000
URL:        http://sources.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=de40ef68

xsystembash: output error messages ourselves

---
 libq/xsystem.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/libq/xsystem.c b/libq/xsystem.c
index e688c5d..199a1d2 100644
--- a/libq/xsystem.c
+++ b/libq/xsystem.c
@@ -27,11 +27,14 @@ static void xsystembash(const char *command)
 
 	default: /* parent */
 		waitpid(p, &status, 0);
-		if (WIFEXITED(status)) {
+		if (WIFSIGNALED(status)) {
+			err("phase crashed with signal %i: %s", WTERMSIG(status),
+			    strsignal(WTERMSIG(status)));
+		} else if (WIFEXITED(status)) {
 			if (WEXITSTATUS(status) == 0)
 				return;
 			else
-				exit(WEXITSTATUS(status));
+				err("phase exited %i", WEXITSTATUS(status));
 		}
 		/* fall through */
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     b990a48b85b71f94323120e50f2ba97958351a97
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Mon Feb 16 11:45:37 2015 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Mon Feb 16 11:45:37 2015 +0000
URL:        http://sources.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=b990a48b

compat: define a S_BLKSIZE fallback value

This define isn't required by POSIX, so some systems omit it.  Define a
fallback value of 512 since that's what everyone generally uses.

URL: https://bugs.gentoo.org/529354

---
 libq/compat.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libq/compat.c b/libq/compat.c
index 928386b..8def2e9 100644
--- a/libq/compat.c
+++ b/libq/compat.c
@@ -51,3 +51,8 @@ char* strcasestr(const char *big, const char *little) {
 #ifdef __hpux
 # define S_BLKSIZE st.st_blksize
 #endif
+
+/* Everyone else */
+#ifndef S_BLKSIZE
+# define S_BLKSIZE 512
+#endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     b0280f87242535cc0e4861c5671c1fa05a6ef141
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Tue Mar 11 04:32:59 2014 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Tue Mar 11 04:32:59 2014 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=b0280f87

vdb: fix fd leaks

Make sure we only open the pkg dir when we don't yet have an fd.

Make sure we close out the fd when we've finished eating a file.

Reported-by: Tim Harder <radhermit <AT> gentoo.org>

---
 libq/vdb.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/libq/vdb.c b/libq/vdb.c
index 44ebf9e..4b7d911 100644
--- a/libq/vdb.c
+++ b/libq/vdb.c
@@ -227,9 +227,11 @@ _q_static q_vdb_pkg_ctx *q_vdb_next_pkg(q_vdb_cat_ctx *cat_ctx)
 _q_static int
 q_vdb_pkg_openat(q_vdb_pkg_ctx *pkg_ctx, const char *file, int flags, mode_t mode)
 {
-	pkg_ctx->fd = openat(pkg_ctx->cat_ctx->fd, pkg_ctx->name, O_RDONLY|O_CLOEXEC|O_PATH);
-	if (pkg_ctx->fd == -1)
-		return -1;
+	if (pkg_ctx->fd == -1) {
+		pkg_ctx->fd = openat(pkg_ctx->cat_ctx->fd, pkg_ctx->name, O_RDONLY|O_CLOEXEC|O_PATH);
+		if (pkg_ctx->fd == -1)
+			return -1;
+	}
 
 	return openat(pkg_ctx->fd, file, flags|O_CLOEXEC, mode);
 }
@@ -259,6 +261,8 @@ q_vdb_pkg_eat(q_vdb_pkg_ctx *pkg_ctx, const char *file, char **bufptr, size_t *b
 	int fd = q_vdb_pkg_openat(pkg_ctx, file, O_RDONLY, 0);
 	bool ret = eat_file_fd(fd, bufptr, buflen);
 	rmspace(*bufptr);
+	if (fd != -1)
+		close(fd);
 	return ret;
 }
 

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     dc0bc1307ad2e8dd550b0a68f3de1d3574be6008
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Tue Feb 18 06:52:17 2014 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Tue Feb 18 06:52:17 2014 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=dc0bc130

xsystem: do not dump when the func exits normally as the package itself wants to fail, not the qmerge process

---
 libq/xsystem.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/libq/xsystem.c b/libq/xsystem.c
index 805eb3f..969d76a 100644
--- a/libq/xsystem.c
+++ b/libq/xsystem.c
@@ -3,7 +3,7 @@
  *
  * Copyright 2005-2010 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
- * $Header: /var/cvsroot/gentoo-projects/portage-utils/libq/xsystem.c,v 1.2 2011/02/21 22:02:59 vapier Exp $
+ * $Header: /var/cvsroot/gentoo-projects/portage-utils/libq/xsystem.c,v 1.3 2014/02/18 06:52:17 vapier Exp $
  */
 
 #include <stdlib.h>
@@ -28,8 +28,12 @@ static void xsystembash(const char *command)
 
 	default: /* parent */
 		waitpid(p, &status, 0);
-		if (WIFEXITED(status) && WEXITSTATUS(status) == 0)
-			return;
+		if (WIFEXITED(status)) {
+			if (WEXITSTATUS(status) == 0)
+				return;
+			else
+				exit(WEXITSTATUS(status));
+		}
 		/* fall through */
 
 	case -1: /* fucked */

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     b73e216c6eeb31222e11f258e1a49fdbad1bf1e7
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Mon Feb 17 06:31:10 2014 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Mon Feb 17 06:31:10 2014 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=b73e216c

revert debug change

---
 libq/vdb.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/libq/vdb.c b/libq/vdb.c
index 1100405..a016c23 100644
--- a/libq/vdb.c
+++ b/libq/vdb.c
@@ -1,7 +1,7 @@
 /*
  * Copyright 2005-2011 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
- * $Header: /var/cvsroot/gentoo-projects/portage-utils/libq/vdb.c,v 1.6 2014/02/16 21:14:24 vapier Exp $
+ * $Header: /var/cvsroot/gentoo-projects/portage-utils/libq/vdb.c,v 1.7 2014/02/17 06:31:10 vapier Exp $
  *
  * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
  * Copyright 2005-2011 Mike Frysinger  - <vapier@gentoo.org>
@@ -196,10 +196,9 @@ _q_static q_vdb_pkg_ctx *q_vdb_open_pkg(q_vdb_cat_ctx *cat_ctx, const char *name
 	q_vdb_pkg_ctx *pkg_ctx;
 	int fd;
 
-//	fd = openat(cat_ctx->fd, name, O_RDONLY|O_CLOEXEC|O_PATH);
+	fd = openat(cat_ctx->fd, name, O_RDONLY|O_CLOEXEC|O_PATH);
 	if (fd == -1)
 		return NULL;
-	fd = -1;
 
 	pkg_ctx = xmalloc(sizeof(*pkg_ctx));
 	pkg_ctx->name = name;

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     1fd24b2037ae8b635ae82f8f8eb767167137759e
Author:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
AuthorDate: Tue Jan  7 19:19:17 2014 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Tue Jan  7 19:19:17 2014 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=1fd24b20

delete O_CLOEXEC fallback logic as porting.h already does this

---
 libq/compat.c | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/libq/compat.c b/libq/compat.c
index 280220f..928386b 100644
--- a/libq/compat.c
+++ b/libq/compat.c
@@ -51,8 +51,3 @@ char* strcasestr(const char *big, const char *little) {
 #ifdef __hpux
 # define S_BLKSIZE st.st_blksize
 #endif
-
-/* Older systems */
-#ifndef O_CLOEXEC
-# define O_CLOEXEC 0
-#endif

[gentoo-commits] proj/portage-utils:master commit in: libq/

[Mike Frysinger]

commit:     39d9c93836e5017794ebeaa763e5f243c4f5daf8
Author:     Fabian Groffen <grobian <AT> gentoo <DOT> org>
AuthorDate: Sun Nov 17 10:26:53 2013 +0000
Commit:     Mike Frysinger <vapier <AT> gentoo <DOT> org>
CommitDate: Sun Nov 17 10:26:53 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/portage-utils.git;a=commit;h=39d9c938

scandirat: copy enough bytes to get the filename

Solaris is a platform without d_reclen.  sizeof(struct dirent) is not
enough to get the whole of d_name contents, since the struct uses
char[1] as workaround to specify a variable size length end of struct
member.

---
 libq/scandirat.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libq/scandirat.c b/libq/scandirat.c
index cd7f272..76a5d4a 100644
--- a/libq/scandirat.c
+++ b/libq/scandirat.c
@@ -1,7 +1,7 @@
 /*
  * Copyright 2005-2011 Gentoo Foundation
  * Distributed under the terms of the GNU General Public License v2
- * $Header: /var/cvsroot/gentoo-projects/portage-utils/libq/scandirat.c,v 1.6 2013/09/29 10:25:25 vapier Exp $
+ * $Header: /var/cvsroot/gentoo-projects/portage-utils/libq/scandirat.c,v 1.7 2013/11/17 10:26:53 grobian Exp $
  *
  * Copyright 2005-2010 Ned Ludd        - <solar@gentoo.org>
  * Copyright 2005-2011 Mike Frysinger  - <vapier@gentoo.org>
@@ -18,7 +18,7 @@
 #if defined(_DIRENT_HAVE_D_RECLEN)
 # define reclen(de) ((de)->d_reclen)
 #else
-# define reclen(de) (sizeof(*(de)))
+# define reclen(de) (sizeof(*(de)) + strlen((de)->d_name))
 #endif
 
 static int scandirat(int dir_fd, const char *dir, struct dirent ***dirlist,