* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-05-03 5:42 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2024-05-03 5:42 UTC (permalink / raw
To: gentoo-commits
commit: 7a5a2f540bc4776121d08d331b141d897eac92f6
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri May 3 05:41:46 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri May 3 05:41:46 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=7a5a2f54
eclean: dist: fix crash with excluded items
Bug: https://bugs.gentoo.org/915055
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index 52ddf2a..94f87f4 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -623,6 +623,7 @@ def doAction(action, options, exclude={}, output=None):
elif not options["quiet"]:
output.einfo("Your " + action + " directory was already clean.")
if saved and not options["quiet"]:
+ verb = "would be"
print()
print(
pp.emph(" The following ")
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-05-03 5:40 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2024-05-03 5:40 UTC (permalink / raw
To: gentoo-commits
commit: 607d7d58638b5f5b085ec861ffb8f751bbe01743
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri May 3 05:25:20 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri May 3 05:25:20 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=607d7d58
eclean: dist: fix typo in comment
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 7f050ee..d38b88e 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -641,8 +641,8 @@ def findPackages(
dead_binpkgs: dict[str, list[str]] = {}
keep_binpkgs = {}
- # FEATURES=pkgdir-index-trusted is now on by default which makes portages inavlids
- # inaccessible
+ # FEATURES=pkgdir-index-trusted is now on by default which makes Portage's
+ # invalids inaccessible
settings = var_dbapi.settings
bin_dbapi = portage.binarytree(pkgdir=pkgdir, settings=settings).dbapi
populate_kwargs = {}
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-05-03 5:40 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2024-05-03 5:40 UTC (permalink / raw
To: gentoo-commits
commit: bb6faf478f7e64d75224e55c3b81deb843dc9749
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri May 3 05:40:05 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri May 3 05:40:05 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=bb6faf47
eclean: dist: fix printing VCS clean items with --quiet
Bug: https://bugs.gentoo.org/928951
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/output.py | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/pym/gentoolkit/eclean/output.py b/pym/gentoolkit/eclean/output.py
index 75f7b02..04fb493 100644
--- a/pym/gentoolkit/eclean/output.py
+++ b/pym/gentoolkit/eclean/output.py
@@ -138,8 +138,11 @@ class OutputControl:
print(self.prettySize(size, True), self.pkg_color(key))
elif self.options["pretend"] or self.options["interactive"]:
# file list mode
- for file_ in clean_list:
- print(file_)
+ if file_type == "checkout":
+ print(clean_list)
+ else:
+ for file_ in clean_list:
+ print(file_)
if self.options["pretend"]:
return False
elif (
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-05-03 5:40 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2024-05-03 5:40 UTC (permalink / raw
To: gentoo-commits
commit: a239275df515741aa20289d7fea067b83ab8daf8
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri May 3 05:25:39 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri May 3 05:26:17 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=a239275d
eclean: dist: fix copyright headers
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/clean.py | 2 +-
pym/gentoolkit/eclean/exclude.py | 2 +-
pym/gentoolkit/eclean/output.py | 2 +-
pym/gentoolkit/eclean/search.py | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/pym/gentoolkit/eclean/clean.py b/pym/gentoolkit/eclean/clean.py
index 0347f1d..918d4dc 100644
--- a/pym/gentoolkit/eclean/clean.py
+++ b/pym/gentoolkit/eclean/clean.py
@@ -1,6 +1,6 @@
#!/usr/bin/python
-# Copyright 2003-2010 Gentoo Foundation
+# Copyright 2003-2023 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
diff --git a/pym/gentoolkit/eclean/exclude.py b/pym/gentoolkit/eclean/exclude.py
index c731ec0..c20574c 100644
--- a/pym/gentoolkit/eclean/exclude.py
+++ b/pym/gentoolkit/eclean/exclude.py
@@ -1,6 +1,6 @@
#!/usr/bin/python
-# Copyright 2003-2010 Gentoo Foundation
+# Copyright 2003-2023 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
diff --git a/pym/gentoolkit/eclean/output.py b/pym/gentoolkit/eclean/output.py
index ebba499..75f7b02 100644
--- a/pym/gentoolkit/eclean/output.py
+++ b/pym/gentoolkit/eclean/output.py
@@ -1,6 +1,6 @@
#!/usr/bin/python
-# Copyright 2003-2010 Gentoo Foundation
+# Copyright 2003-2023 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index d38b88e..aaa67dc 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -1,6 +1,6 @@
#!/usr/bin/python
-# Copyright 2003-2010 Gentoo Foundation
+# Copyright 2003-2024 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-05-03 5:11 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2024-05-03 5:11 UTC (permalink / raw
To: gentoo-commits
commit: 7405fdccd73364662089f1001231bf8c24916ca3
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri May 3 05:10:41 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri May 3 05:10:41 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=7405fdcc
eclean: dist: fix operating on an empty directory
Zac pointed out that not all return paths had been updated.
Bug: https://bugs.gentoo.org/928138
Fixes: 87912b460c533f8b010617018fdad42246db85f0
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index b696b63..7f050ee 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -115,10 +115,11 @@ class DistfilesSearch:
pkgs = {}
saved = {}
deprecated = {}
+ vcs = {}
installed_included = False
# Check if DISTDIR is empty, return early
if not os.listdir(_distdir):
- return clean_me, saved, deprecated
+ return clean_me, saved, deprecated, vcs
# create a big CPV->SRC_URI dict of packages
# whose distfiles should be kept
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-02-18 2:19 Brian Dolbec
0 siblings, 0 replies; 43+ messages in thread
From: Brian Dolbec @ 2024-02-18 2:19 UTC (permalink / raw
To: gentoo-commits
commit: f410ba4ae94c801564f5b0bd3003e429adf2b12e
Author: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
AuthorDate: Sat Feb 17 02:34:39 2024 +0000
Commit: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
CommitDate: Sun Feb 18 02:16:28 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=f410ba4a
eclean/search.py: Fix find_packages docstring for invalid_paths
Signed-off-by: Brian Dolbec <dolsen <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 47df3a1..b696b63 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -591,7 +591,6 @@ def findPackages(
) -> tuple[dict[str, list[str]], dict[str, list[str]]]:
"""Find obsolete binary packages.
- @param invalid_paths:
@param options: dict of options determined at runtime
@type options: dict
@param exclude: exclusion dict (as defined in the exclude.parseExcludeFile class)
@@ -610,8 +609,8 @@ def findPackages(
@param var_dbapi: defaults to portage.db[portage.root]["vartree"].dbapi
Can be overridden for tests.
- @return binary packages to remove. e.g. {'cat/pkg-ver': [filepath]}
- @rtype: dict
+ @return binary packages to remove. e.g. {'cat/pkg-ver': [filepath]}, invalid_paths
+ @rtype: dict, dict
"""
if exclude is None:
exclude = {}
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-02-18 2:19 Brian Dolbec
0 siblings, 0 replies; 43+ messages in thread
From: Brian Dolbec @ 2024-02-18 2:19 UTC (permalink / raw
To: gentoo-commits
commit: 70cb55fa5fd2af7c7e46c94dc423a96bbedd83a5
Author: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
AuthorDate: Sun Feb 4 21:00:49 2024 +0000
Commit: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
CommitDate: Sun Feb 18 02:16:03 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=70cb55fa
eclean: Handle InvalidDepstring info in _deps_equal
Add try/except pair to _deps_equal() to output relavent details
causing the exception in order to aid the user to fix the issue.
Mark binpkg dep failures as a non match for possible deletion.
Make the ebuild dep failure a warning only, return True to save
the binpkg.
Add parameter docstring info
Bug: https://bugs.gentoo.org/923439
Signed-off-by: Brian Dolbec <dolsen <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 54 ++++++++++++++++++++++++++++++++++++-----
1 file changed, 48 insertions(+), 6 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 2eebcfd..47df3a1 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -17,6 +17,7 @@ import portage
from portage.dep import Atom, use_reduce
from portage.dep._slot_operator import strip_slots
from portage.dep.libc import find_libc_deps, strip_libc_deps
+from portage.exception import InvalidDependString
import gentoolkit.pprinter as pp
from gentoolkit.eclean.exclude import (
@@ -526,13 +527,51 @@ class DistfilesSearch:
return clean_me, saved
-def _deps_equal(deps_a, eapi_a, deps_b, eapi_b, libc_deps, uselist=None):
- """Compare two dependency lists given a set of USE flags"""
+def _deps_equal(deps_a, eapi_a, deps_b, eapi_b, libc_deps, uselist=None, cpv=None):
+ """Compare two dependency lists given a set of USE flags
+
+ @param deps_a: binpkg DEPEND string (for InvalidDependString errors)
+ @rtype: string
+ @param eapi_a: EAPI
+ @rtype: string
+ @param deps_b: ebuild DEPEND string (for InvalidDependString errors)
+ @rtype: string
+ @param eapi_b: EAPI
+ @rtype: string
+ @param libc_deps: List of libc packages (or atoms if realized is passed).
+ @rtype: list
+ @param uselist: use flag list
+ @rtype: frozenset
+ @param cpv: Cat/Pkg-version
+ @rtype: string
+ """
if deps_a == deps_b:
return True
+ try:
+ deps_a = use_reduce(deps_a, uselist=uselist, eapi=eapi_a, token_class=Atom)
+ except InvalidDependString: # the binpkg depend string is bad
+ print(
+ pp.warn(
+ "Warning: Invalid binpkg DEPEND string found for: %s, %s"
+ " | tagging for removal" % (cpv, deps_a)
+ ),
+ file=sys.stderr,
+ )
+ return False
+ try:
+ deps_b = use_reduce(deps_b, uselist=uselist, eapi=eapi_b, token_class=Atom)
+ except InvalidDependString as er: # the ebuild depend string is bad
+ print(
+ pp.warn("Warning: Invalid ebuild DEPEND String found for: %s" % cpv),
+ file=sys.stderr,
+ )
+ print(
+ pp.warn("Warning: DEPEND string for ebuild: %s" % deps_b),
+ file=sys.stderr,
+ )
+ print(er, file=sys.stderr)
+ return True
- deps_a = use_reduce(deps_a, uselist=uselist, eapi=eapi_a, token_class=Atom)
- deps_b = use_reduce(deps_b, uselist=uselist, eapi=eapi_b, token_class=Atom)
strip_libc_deps(deps_a, libc_deps)
strip_libc_deps(deps_b, libc_deps)
strip_slots(deps_a)
@@ -656,13 +695,16 @@ def findPackages(
binpkg_metadata = dict(zip(keys, bin_dbapi.aux_get(cpv, keys)))
ebuild_metadata = dict(zip(keys, port_dbapi.aux_get(cpv, keys)))
+ deps_binpkg = " ".join(binpkg_metadata[key] for key in dep_keys)
+ deps_ebuild = " ".join(ebuild_metadata[key] for key in dep_keys)
if _deps_equal(
- " ".join(binpkg_metadata[key] for key in dep_keys),
+ deps_binpkg,
binpkg_metadata["EAPI"],
- " ".join(ebuild_metadata[key] for key in dep_keys),
+ deps_ebuild,
ebuild_metadata["EAPI"],
libc_deps,
frozenset(binpkg_metadata["USE"].split()),
+ cpv,
):
continue
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-01-19 7:03 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2024-01-19 7:03 UTC (permalink / raw
To: gentoo-commits
commit: bd9c485781584e951e703ee191308db23506af9a
Author: Siddhanth Rathod <xsiddhanthrathod <AT> gmail <DOT> com>
AuthorDate: Fri Jan 19 06:59:55 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Jan 19 07:03:39 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=bd9c4857
eclean: handle when git3-src doesn't exist (port to Pathlib)
Followup to c584d83705a2ca08961e4f0b541442fdf9a75947.
Bug: https://bugs.gentoo.org/922455
Signed-off-by: Siddhanth Rathod <xsiddhanthrathod <AT> gmail.com>
Closes: https://github.com/gentoo/gentoolkit/pull/41
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 50 ++++++++++++++++++++---------------------
1 file changed, 25 insertions(+), 25 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 3610920..2eebcfd 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -5,12 +5,13 @@
import os
+import shlex
import stat
import sys
-import shlex
from functools import partial
from inspect import signature
-from typing import Optional
+from pathlib import Path
+from typing import Optional, Set
import portage
from portage.dep import Atom, use_reduce
@@ -136,7 +137,7 @@ class DistfilesSearch:
# gather the files to be cleaned
self.output("...checking limits for %d ebuild sources" % len(pkgs))
- vcs = self.vcs_check(_distdir)
+ vcs = self.vcs_check(Path(_distdir))
checks = self._get_default_checks(size_limit, time_limit, exclude, destructive)
checks.extend(extra_checks)
clean_me = self._check_limits(_distdir, checks, clean_me)
@@ -335,31 +336,30 @@ class DistfilesSearch:
deprecated.update(_deprecated)
return pkgs, deprecated
- def vcs_check(self, distdir):
+ def vcs_check(self, distdir: Path) -> Set:
"""Checks $DISTDIR/vcs-src for checkouts which are not in the vardb"""
# For now we only check git
- vcs_src = os.path.join(distdir, "git3-src")
- if not os.path.exists(vcs_src):
- return {}
-
+ vcs_src = distdir / "git3-src"
expected_dirs = set()
- for i in set(self.vardb.cpv_all()):
- if "live" in self.vardb.aux_get(i, ["PROPERTIES"]):
- try:
- # try to get the dir names of the cloned
- # repos from the environment file.
- vcs_dir = {
- i.split("=")[-1].strip('"')
- for i in shlex.split(
- self.vardb._aux_env_search(i, ["EVCS_STORE_DIRS"])[
- "EVCS_STORE_DIRS"
- ].strip("()")
- )
- }
- expected_dirs.update(vcs_dir)
- except KeyError:
- pass
- actual_dirs = {os.path.join(vcs_src, i) for i in os.listdir(vcs_src)}
+ actual_dirs = set()
+ if vcs_src.is_dir():
+ for i in set(self.vardb.cpv_all()):
+ if "live" in self.vardb.aux_get(i, ["PROPERTIES"]):
+ try:
+ # try to get the dir names of the cloned
+ # repos from the environment file.
+ vcs_dir = {
+ i.split("=")[-1].strip('"')
+ for i in shlex.split(
+ self.vardb._aux_env_search(i, ["EVCS_STORE_DIRS"])[
+ "EVCS_STORE_DIRS"
+ ].strip("()")
+ )
+ }
+ expected_dirs.update(vcs_dir)
+ except KeyError:
+ pass
+ actual_dirs = {str(i) for i in vcs_src.iterdir() if i.is_dir()}
return actual_dirs.difference(expected_dirs)
def _fetch_restricted(self, pkgs_, cpvs):
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-01-19 6:15 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2024-01-19 6:15 UTC (permalink / raw
To: gentoo-commits
commit: c584d83705a2ca08961e4f0b541442fdf9a75947
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri Jan 19 06:14:43 2024 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Jan 19 06:15:12 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=c584d837
eclean: fix eclean-dist with no git3-src dir
Bug: https://bugs.gentoo.org/922455
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index f03c521..3610920 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -339,6 +339,9 @@ class DistfilesSearch:
"""Checks $DISTDIR/vcs-src for checkouts which are not in the vardb"""
# For now we only check git
vcs_src = os.path.join(distdir, "git3-src")
+ if not os.path.exists(vcs_src):
+ return {}
+
expected_dirs = set()
for i in set(self.vardb.cpv_all()):
if "live" in self.vardb.aux_get(i, ["PROPERTIES"]):
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2024-01-12 19:16 Matt Turner
0 siblings, 0 replies; 43+ messages in thread
From: Matt Turner @ 2024-01-12 19:16 UTC (permalink / raw
To: gentoo-commits
commit: a3e56252cdde2f4fb51766c580c0137f28843510
Author: Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Thu Jan 11 21:31:12 2024 +0000
Commit: Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Fri Jan 12 19:15:28 2024 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=a3e56252
eclean: Strip libc dependencies from --changed-deps calculations
Closes: https://bugs.gentoo.org/921679
Closes: https://github.com/gentoo/gentoolkit/pull/38
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index de424c5..f03c521 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -15,6 +15,7 @@ from typing import Optional
import portage
from portage.dep import Atom, use_reduce
from portage.dep._slot_operator import strip_slots
+from portage.dep.libc import find_libc_deps, strip_libc_deps
import gentoolkit.pprinter as pp
from gentoolkit.eclean.exclude import (
@@ -522,13 +523,15 @@ class DistfilesSearch:
return clean_me, saved
-def _deps_equal(deps_a, eapi_a, deps_b, eapi_b, uselist=None):
+def _deps_equal(deps_a, eapi_a, deps_b, eapi_b, libc_deps, uselist=None):
"""Compare two dependency lists given a set of USE flags"""
if deps_a == deps_b:
return True
deps_a = use_reduce(deps_a, uselist=uselist, eapi=eapi_a, token_class=Atom)
deps_b = use_reduce(deps_b, uselist=uselist, eapi=eapi_b, token_class=Atom)
+ strip_libc_deps(deps_a, libc_deps)
+ strip_libc_deps(deps_b, libc_deps)
strip_slots(deps_a)
strip_slots(deps_b)
return deps_a == deps_b
@@ -583,6 +586,8 @@ def findPackages(
print(pp.error("Error: %s" % str(er)), file=sys.stderr)
exit(1)
+ libc_deps = find_libc_deps(var_dbapi, False)
+
# Create a dictionary of all installed packages
if destructive and package_names:
installed = dict.fromkeys(var_dbapi.cp_all())
@@ -653,6 +658,7 @@ def findPackages(
binpkg_metadata["EAPI"],
" ".join(ebuild_metadata[key] for key in dep_keys),
ebuild_metadata["EAPI"],
+ libc_deps,
frozenset(binpkg_metadata["USE"].split()),
):
continue
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-12-03 7:51 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-12-03 7:51 UTC (permalink / raw
To: gentoo-commits
commit: 87912b460c533f8b010617018fdad42246db85f0
Author: Siddhanth Rathod <xsiddhanthrathod <AT> gmail <DOT> com>
AuthorDate: Thu Sep 21 20:20:15 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sun Dec 3 07:51:44 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=87912b46
Enhancement: eclean-dist handle git checkouts
A new feature for eclean-dist to clean git3-src.
Optionally, cleaning the vcs-src can be skipped with --skip-vcs.
Bug: https://bugs.gentoo.org/622938
Signed-off-by: Siddhanth Rathod <xsiddhanthrathod <AT> gmail.com>
Closes: https://github.com/gentoo/gentoolkit/pull/33
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/clean.py | 35 ++++++++++++++++++++++++++++++++--
pym/gentoolkit/eclean/cli.py | 42 ++++++++++++++++++++++++++++-------------
pym/gentoolkit/eclean/search.py | 28 ++++++++++++++++++++++++++-
3 files changed, 89 insertions(+), 16 deletions(-)
diff --git a/pym/gentoolkit/eclean/clean.py b/pym/gentoolkit/eclean/clean.py
index 92449b4..0347f1d 100644
--- a/pym/gentoolkit/eclean/clean.py
+++ b/pym/gentoolkit/eclean/clean.py
@@ -5,6 +5,7 @@
import os
+import shutil
import sys
import gentoolkit.pprinter as pp
@@ -25,7 +26,7 @@ class CleanUp:
self.controller = controller
self.quiet = quiet
- def clean_dist(self, clean_dict):
+ def clean_dist(self, clean_dict, vcs):
"""Calculate size of each entry for display, prompt user if needed,
delete files if approved and return the total size of files that
have been deleted.
@@ -41,6 +42,7 @@ class CleanUp:
for key in sorted(clean_dict):
clean_size += self._clean_files(clean_dict[key], key, file_type)
# return total size of deleted or to delete files
+ clean_size += self._clean_vcs_src(vcs)
return clean_size
def clean_pkgs(self, clean_dict, pkgdir):
@@ -74,7 +76,7 @@ class CleanUp:
# return total size of deleted or to delete files
return clean_size
- def pretend_clean(self, clean_dict):
+ def pretend_clean(self, clean_dict, vcs={}):
"""Shortcut function that calculates total space savings
for the files in clean_dict.
@@ -85,6 +87,8 @@ class CleanUp:
file_type = "file"
clean_size = 0
# tally all entries one by one; sorting helps reading
+ if vcs:
+ clean_size += self._clean_vcs_src(vcs, pretend=True)
for key in sorted(clean_dict):
key_size = self._get_size(clean_dict[key])
self.controller(key_size, key, clean_dict[key], file_type)
@@ -153,3 +157,30 @@ class CleanUp:
print(pp.error("Could not delete " + file_), file=sys.stderr)
print(pp.error("Error: %s" % str(er)), file=sys.stderr)
return clean_size
+
+ def _clean_vcs_src(self, deprecated_vcs, pretend=False):
+ clean_size = 0
+ for checkout in deprecated_vcs:
+ csize = 0
+ for path, dirs, files in os.walk(checkout):
+ for f in files:
+ fp = os.path.join(path, f)
+ try:
+ statinfo = os.stat(fp)
+ except OSError as er:
+ print(
+ pp.error("Could not get stat info for:" + fp),
+ file=sys.stderr,
+ )
+ print(pp.error("Error: %s" % str(er)), file=sys.stderr)
+ clean_size += statinfo.st_size
+ csize += statinfo.st_size
+
+ try:
+ self.controller(csize, checkout, checkout, "checkout")
+ if not pretend:
+ shutil.rmtree(checkout)
+ except OSError as er:
+ print(pp.error("Could not delete " + checkout), file=sys.stderr)
+ print(pp.error("Error: %s" % str(er)), file=sys.stderr)
+ return clean_size
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index e3dd540..52ddf2a 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -13,25 +13,25 @@ __version__ = "@VERSION@"
__productname__ = "eclean"
__description__ = "A cleaning tool for Gentoo distfiles and binaries."
+import getopt
import os
-import sys
import re
+import sys
import time
-import getopt
import portage
-from portage.output import white, yellow, turquoise, green, red
+from portage.output import green, red, turquoise, white, yellow
import gentoolkit.pprinter as pp
+from gentoolkit.eclean.clean import CleanUp
+from gentoolkit.eclean.exclude import ParseExcludeFileException, parseExcludeFile
+from gentoolkit.eclean.output import OutputControl
from gentoolkit.eclean.search import (
DistfilesSearch,
findPackages,
- port_settings,
pkgdir,
+ port_settings,
)
-from gentoolkit.eclean.exclude import parseExcludeFile, ParseExcludeFileException
-from gentoolkit.eclean.clean import CleanUp
-from gentoolkit.eclean.output import OutputControl
# from gentoolkit.eclean.dbapi import Dbapi
from gentoolkit.eprefix import EPREFIX
@@ -309,6 +309,10 @@ def printUsage(_error=None, help=None, unresolved_invalids=None):
" " + '"two hundreds kilobytes", etc. Units are: ' + "G, M, K and B.",
file=out,
)
+ print(
+ yellow(" --skip-vcs") + " - skip cleaning of vcs_src ",
+ file=out,
+ )
print(file=out)
print(
"More detailed instruction can be found in",
@@ -420,6 +424,8 @@ def parseArgs(options={}):
options["unique-use"] = True
elif o in ("--no-clean-invalid"):
options["no-clean-invalid"] = True
+ elif o in ("--skip-vcs"):
+ options["skip-vcs"] = True
else:
return_code = False
# sanity check of --deep only options:
@@ -457,7 +463,11 @@ def parseArgs(options={}):
"verbose",
]
getopt_options["short"]["distfiles"] = "fs:"
- getopt_options["long"]["distfiles"] = ["fetch-restricted", "size-limit="]
+ getopt_options["long"]["distfiles"] = [
+ "fetch-restricted",
+ "size-limit=",
+ "skip-vcs",
+ ]
getopt_options["short"]["packages"] = "iu"
getopt_options["long"]["packages"] = [
"ignore-failure",
@@ -481,6 +491,7 @@ def parseArgs(options={}):
options["ignore-failure"] = False
options["no-clean-invalid"] = False
options["unique-use"] = False
+ options["skip-vcs"] = False
# if called by a well-named symlink, set the action accordingly:
action = None
# temp print line to ensure it is the svn/branch code running, etc..
@@ -546,6 +557,7 @@ def doAction(action, options, exclude={}, output=None):
files_type = "distfiles"
saved = {}
deprecated = {}
+ vcs = []
# find files to delete, depending on the action
if not options["quiet"]:
output.einfo("Building file list for " + action + " cleaning...")
@@ -567,7 +579,7 @@ def doAction(action, options, exclude={}, output=None):
# portdb=Dbapi(portage.db[portage.root]["porttree"].dbapi),
# var_dbapi=Dbapi(portage.db[portage.root]["vartree"].dbapi),
)
- clean_me, saved, deprecated = engine.findDistfiles(
+ clean_me, saved, deprecated, vcs = engine.findDistfiles(
exclude=exclude,
destructive=options["destructive"],
fetch_restricted=options["fetch-restricted"],
@@ -581,7 +593,7 @@ def doAction(action, options, exclude={}, output=None):
cleaner = CleanUp(output.progress_controller, options["quiet"])
# actually clean files if something was found
- if clean_me:
+ if clean_me or vcs:
# verbose pretend message
if options["pretend"] and not options["quiet"]:
output.einfo("Here are the " + files_type + " that would be deleted:")
@@ -590,9 +602,13 @@ def doAction(action, options, exclude={}, output=None):
output.einfo("Cleaning " + files_type + "...")
# do the cleanup, and get size of deleted files
if options["pretend"]:
- clean_size = cleaner.pretend_clean(clean_me)
+ if options["skip-vcs"]:
+ vcs = {}
+ clean_size = cleaner.pretend_clean(clean_me, vcs)
elif action in ["distfiles"]:
- clean_size = cleaner.clean_dist(clean_me)
+ if options["skip-vcs"]:
+ vcs = {}
+ clean_size = cleaner.clean_dist(clean_me, vcs)
elif action in ["packages"]:
clean_size = cleaner.clean_pkgs(clean_me, pkgdir)
# vocabulary for final message
@@ -602,7 +618,7 @@ def doAction(action, options, exclude={}, output=None):
verb = "were"
# display freed space
if not options["quiet"]:
- output.total("normal", clean_size, len(clean_me), verb, action)
+ output.total("normal", clean_size, len(clean_me) + len(vcs), verb, action)
# nothing was found
elif not options["quiet"]:
output.einfo("Your " + action + " directory was already clean.")
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 77f16af..de424c5 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -7,6 +7,7 @@
import os
import stat
import sys
+import shlex
from functools import partial
from inspect import signature
from typing import Optional
@@ -134,6 +135,7 @@ class DistfilesSearch:
# gather the files to be cleaned
self.output("...checking limits for %d ebuild sources" % len(pkgs))
+ vcs = self.vcs_check(_distdir)
checks = self._get_default_checks(size_limit, time_limit, exclude, destructive)
checks.extend(extra_checks)
clean_me = self._check_limits(_distdir, checks, clean_me)
@@ -148,7 +150,7 @@ class DistfilesSearch:
+ "%s remaining candidates to clean" % len(clean_me)
)
clean_me, saved = self._check_excludes(exclude, clean_me)
- return clean_me, saved, deprecated
+ return clean_me, saved, deprecated, vcs
# begin _check_limits code block
@@ -332,6 +334,30 @@ class DistfilesSearch:
deprecated.update(_deprecated)
return pkgs, deprecated
+ def vcs_check(self, distdir):
+ """Checks $DISTDIR/vcs-src for checkouts which are not in the vardb"""
+ # For now we only check git
+ vcs_src = os.path.join(distdir, "git3-src")
+ expected_dirs = set()
+ for i in set(self.vardb.cpv_all()):
+ if "live" in self.vardb.aux_get(i, ["PROPERTIES"]):
+ try:
+ # try to get the dir names of the cloned
+ # repos from the environment file.
+ vcs_dir = {
+ i.split("=")[-1].strip('"')
+ for i in shlex.split(
+ self.vardb._aux_env_search(i, ["EVCS_STORE_DIRS"])[
+ "EVCS_STORE_DIRS"
+ ].strip("()")
+ )
+ }
+ expected_dirs.update(vcs_dir)
+ except KeyError:
+ pass
+ actual_dirs = {os.path.join(vcs_src, i) for i in os.listdir(vcs_src)}
+ return actual_dirs.difference(expected_dirs)
+
def _fetch_restricted(self, pkgs_, cpvs):
"""perform fetch restricted non-destructive source
filename lookups
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-10-08 3:28 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-10-08 3:28 UTC (permalink / raw
To: gentoo-commits
commit: a12d24f99d03c3c4975d84d63083449cd8487ef9
Author: Siddhanth Rathod <xsiddhanthrathod <AT> gmail <DOT> com>
AuthorDate: Thu Oct 5 13:34:59 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sun Oct 8 03:27:53 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=a12d24f9
eclean: pkg: suppress invalid binary error
[sam: Note that this needs https://github.com/gentoo/portage/pull/1123 on
the Portage side.]
Bug: https://bugs.gentoo.org/900224
Signed-off-by: Siddhanth Rathod <xsiddhanthrathod <AT> gmail.com>
Closes: https://github.com/gentoo/gentoolkit/pull/35
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index ce8fe37..77f16af 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -572,8 +572,11 @@ def findPackages(
# inaccessible
settings = var_dbapi.settings
bin_dbapi = portage.binarytree(pkgdir=pkgdir, settings=settings).dbapi
+ populate_kwargs = {}
+ if "invalid_errors" in signature(bin_dbapi.bintree.populate).parameters:
+ populate_kwargs["invalid_errors"] = False
if "force_reindex" in signature(bin_dbapi.bintree.populate).parameters:
- bin_dbapi.bintree.populate(force_reindex=True)
+ bin_dbapi.bintree.populate(force_reindex=True, **populate_kwargs)
for cpv in bin_dbapi.cpv_all():
cp = portage.cpv_getkey(cpv)
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-10-08 3:28 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-10-08 3:28 UTC (permalink / raw
To: gentoo-commits
commit: c846c08c046588a98f2b40a45ac74b84cda21a89
Author: Siddhanth Rathod <xsiddhanthrathod <AT> gmail <DOT> com>
AuthorDate: Thu Oct 5 13:29:32 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Sun Oct 8 03:27:35 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=c846c08c
eclean: pkg: fix pkgindex handling
Last implemention was importing binhost module incorrectly:
calling Modules() from portage.module which sets binhost var to the BinhostHandler class
which would always fail as it needs name and namepath, leading to always call emaint
via a subprocess call.
Signed-off-by: Siddhanth Rathod <xsiddhanthrathod <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/clean.py | 17 +++++---
pym/gentoolkit/eclean/pkgindex.py | 86 ---------------------------------------
2 files changed, 11 insertions(+), 92 deletions(-)
diff --git a/pym/gentoolkit/eclean/clean.py b/pym/gentoolkit/eclean/clean.py
index 87d7aac..92449b4 100644
--- a/pym/gentoolkit/eclean/clean.py
+++ b/pym/gentoolkit/eclean/clean.py
@@ -8,7 +8,9 @@ import os
import sys
import gentoolkit.pprinter as pp
-from gentoolkit.eclean.pkgindex import PkgIndex
+import portage
+from portage.emaint.main import TaskHandler
+from portage.emaint.modules.binhost import binhost
class CleanUp:
@@ -61,11 +63,14 @@ class CleanUp:
# run 'emaint --fix' here
if clean_size:
- index_control = PkgIndex(self.controller)
- # emaint is not yet importable so call it
- # print a blank line here for separation
- print()
- clean_size += index_control.clean_pkgs_index(self.quiet)
+ file = os.path.join(portage.settings["PKGDIR"], "Packages")
+ size1 = os.stat(file).st_size
+ TaskHandler(show_progress_bar=self.quiet).run_tasks(
+ [binhost.BinhostHandler], "fix"
+ )
+ size = size1 - os.stat(file).st_size
+ self.controller(size, "Packages Index", file, "Index")
+ clean_size += size
# return total size of deleted or to delete files
return clean_size
diff --git a/pym/gentoolkit/eclean/pkgindex.py b/pym/gentoolkit/eclean/pkgindex.py
deleted file mode 100644
index 7d6fade..0000000
--- a/pym/gentoolkit/eclean/pkgindex.py
+++ /dev/null
@@ -1,86 +0,0 @@
-#!/usr/bin/python
-
-# Copyright 2003-2010 Gentoo Foundation
-# Distributed under the terms of the GNU General Public License v2
-
-import subprocess
-import os
-import sys
-
-import gentoolkit.pprinter as pp
-from gentoolkit.eprefix import EPREFIX
-
-import portage
-from portage.module import (
- InvalidModuleName,
- Modules,
-)
-from portage.emaint.main import TaskHandler
-
-
-class PkgIndex:
- """Handle the cleaning of the binpkg Package
- Index file
-
- @type output: class
- @param output: optional output class for printing
- """
-
- def __init__(self, controller=None):
- self.controller = controller
- # backup command line call
- self.emaint_cmd = "%s/usr/sbin/emaint --fix binhost" % EPREFIX
-
- def _get_emaint_binhost(self):
- """Obtain a reference to the binhost module class
-
- @sets: self.binhost to BinhostHandler class
- @rtype: boolean
- """
- try:
- self.emaint_control = Modules()
- self.binhost = self.emaint_control._get_class("binhost")
- except InvalidModuleName as er:
- print(pp.error("Error importing emaint binhost module"), file=sys.stderr)
- print(pp.error("Original error: " + er), file=sys.stderr)
- except:
- return False
- return True
-
- def clean_pkgs_index(self, quiet):
- """This will clean the binpkgs packages index file
-
- @param quiet: boolean
- @return: the difference in file size
- """
- file_ = os.path.join(portage.settings["PKGDIR"], "Packages")
- statinfo = os.stat(file_)
- size1 = statinfo.st_size
- show_progress = not quiet
- if self._get_emaint_binhost():
- self.taskmaster = TaskHandler(show_progress_bar=show_progress)
- tasks = [self.binhost]
- self.taskmaster.run_tasks(tasks)
- else:
- self.call_emaint()
- statinfo = os.stat(file_)
- clean_size = size1 - statinfo.st_size
- self.controller(clean_size, "Packages Index", file_, "Index")
- return clean_size
-
- def call_emaint(self):
- """Run the stand alone emaint script from
- a subprocess call.
-
- @rtype: integer
- @return: the difference in file size
- """
- try:
- retcode = subprocess.call(self.emaint_cmd, shell=True)
- if retcode < 0:
- print(
- pp.error("Child was terminated by signal" + str(-retcode)),
- file=sys.stderr,
- )
- except OSError as e:
- print(pp.error("Execution failed:" + e), file=sys.stderr)
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-09-29 7:20 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-09-29 7:20 UTC (permalink / raw
To: gentoo-commits
commit: e49cc3493ab77678a740bf8a27cf22e77e061409
Author: Siddhanth Rathod <xsiddhanthrathod <AT> gmail <DOT> com>
AuthorDate: Thu Sep 28 18:54:44 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Fri Sep 29 07:20:18 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=e49cc349
eclean-pkg: fix compatibility with FEATURES=pkgdir-index-trusted
Portage recently enabled FEATURES=pkgdir-index-trusted by default which breaks
the eclean-invalids feature.
Force reindexing after cleaning up to fix this.
See-also: https://github.com/gentoo/portage/commit/3bc7bfef51d88fd716e882d3931f2873e3493e01
Bug: https://bugs.gentoo.org/889300
Bug: https://bugs.gentoo.org/900224
Signed-off-by: Siddhanth Rathod <xsiddhanthrathod <AT> gmail.com>
Closes: https://github.com/gentoo/gentoolkit/pull/34
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index a2ac0ce..ce8fe37 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -8,6 +8,7 @@ import os
import stat
import sys
from functools import partial
+from inspect import signature
from typing import Optional
import portage
@@ -16,13 +17,12 @@ from portage.dep._slot_operator import strip_slots
import gentoolkit.pprinter as pp
from gentoolkit.eclean.exclude import (
- exclDictMatchCP,
exclDictExpand,
exclDictExpandPkgname,
+ exclDictMatchCP,
exclMatchFilename,
)
-
# Misc. shortcuts to some portage stuff:
port_settings = portage.settings
pkgdir = port_settings["PKGDIR"]
@@ -568,7 +568,12 @@ def findPackages(
dead_binpkgs: dict[str, list[str]] = {}
keep_binpkgs = {}
- bin_dbapi = portage.binarytree(pkgdir=pkgdir, settings=var_dbapi.settings).dbapi
+ # FEATURES=pkgdir-index-trusted is now on by default which makes portages inavlids
+ # inaccessible
+ settings = var_dbapi.settings
+ bin_dbapi = portage.binarytree(pkgdir=pkgdir, settings=settings).dbapi
+ if "force_reindex" in signature(bin_dbapi.bintree.populate).parameters:
+ bin_dbapi.bintree.populate(force_reindex=True)
for cpv in bin_dbapi.cpv_all():
cp = portage.cpv_getkey(cpv)
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-08-24 19:52 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-08-24 19:52 UTC (permalink / raw
To: gentoo-commits
commit: 9ea9cfe27f3debe4f97f3b04da6681fc3ae11b88
Author: Siddhanth Rathod <xsiddhanthrathod <AT> gmail <DOT> com>
AuthorDate: Thu Aug 24 15:41:35 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Thu Aug 24 19:52:20 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=9ea9cfe2
eclean: Make cleaning invalid default
Bug: https://bugs.gentoo.org/900224
Signed-off-by: Siddhanth Rathod <xsiddhanthrathod <AT> gmail.com>
Closes: https://github.com/gentoo/gentoolkit/pull/31
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index 75c5c63..b180641 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -262,7 +262,8 @@ def printUsage(_error=None, help=None, unresolved_invalids=None):
file=out,
)
print(
- yellow(" -N, --clean-invalid") + " - cleanup invalid binpkgs",
+ yellow(" --no-clean-invalid")
+ + " - Skip cleaning invalid binpkgs",
file=out,
)
print(
@@ -417,8 +418,8 @@ def parseArgs(options={}):
options["ignore-failure"] = True
elif o in ("-u", "--unique-use"):
options["unique-use"] = True
- elif o in ("-N", "--skip-invalid"):
- options["clean-invalid"] = False
+ elif o in ("--no-clean-invalid"):
+ options["no-clean-invalid"] = True
else:
return_code = False
# sanity check of --deep only options:
@@ -457,12 +458,12 @@ def parseArgs(options={}):
]
getopt_options["short"]["distfiles"] = "fs:"
getopt_options["long"]["distfiles"] = ["fetch-restricted", "size-limit="]
- getopt_options["short"]["packages"] = "iNu"
+ getopt_options["short"]["packages"] = "iu"
getopt_options["long"]["packages"] = [
"ignore-failure",
"changed-deps",
- "clean-invalid",
"unique-use",
+ "no-clean-invalid",
]
# set default options, except 'nocolor', which is set in main()
options["interactive"] = False
@@ -478,7 +479,7 @@ def parseArgs(options={}):
options["verbose"] = False
options["changed-deps"] = False
options["ignore-failure"] = False
- options["clean-invalid"] = False
+ options["no-clean-invalid"] = False
options["unique-use"] = False
# if called by a well-named symlink, set the action accordingly:
action = None
@@ -625,7 +626,7 @@ def doAction(action, options, exclude={}, output=None):
output.set_colors("deprecated")
output.list_pkgs(deprecated)
if action in ["packages"]:
- if invalids and options["clean-invalid"]:
+ if invalids and not options["no-clean-invalid"]:
if type(invalids) == list:
printUsage(_error="invalid_paths", unresolved_invalids=invalids)
sys.exit(1)
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-08-22 20:54 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-08-22 20:54 UTC (permalink / raw
To: gentoo-commits
commit: 62ae253a36d77ac238f1dbc47e4757b92456b174
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 22 20:54:01 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Tue Aug 22 20:54:01 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=62ae253a
eclean-pkg: placate black
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index adc4210..75c5c63 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -262,8 +262,7 @@ def printUsage(_error=None, help=None, unresolved_invalids=None):
file=out,
)
print(
- yellow(" -N, --clean-invalid")
- + " - cleanup invalid binpkgs",
+ yellow(" -N, --clean-invalid") + " - cleanup invalid binpkgs",
file=out,
)
print(
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-08-22 20:52 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-08-22 20:52 UTC (permalink / raw
To: gentoo-commits
commit: d40b5009c1f73730ad4d68169b1302393cdce8cf
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 22 20:52:35 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Tue Aug 22 20:52:35 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=d40b5009
eclean-pkg: fix --help alignment, mention shortopt for clean-invalid
Bug: https://bugs.gentoo.org/900224
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index 7591959..adc4210 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -262,8 +262,8 @@ def printUsage(_error=None, help=None, unresolved_invalids=None):
file=out,
)
print(
- yellow(" --clean-invalid")
- + " - cleanup invalid binpkgs",
+ yellow(" -N, --clean-invalid")
+ + " - cleanup invalid binpkgs",
file=out,
)
print(
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-08-22 20:50 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-08-22 20:50 UTC (permalink / raw
To: gentoo-commits
commit: 2aae0531f51064651240a5957bf724f30f5c0cd1
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 22 20:49:41 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Tue Aug 22 20:50:14 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=2aae0531
eclean-pkg: fix --unique-use shortopt
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index aa8b20f..7591959 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -458,7 +458,7 @@ def parseArgs(options={}):
]
getopt_options["short"]["distfiles"] = "fs:"
getopt_options["long"]["distfiles"] = ["fetch-restricted", "size-limit="]
- getopt_options["short"]["packages"] = "iN"
+ getopt_options["short"]["packages"] = "iNu"
getopt_options["long"]["packages"] = [
"ignore-failure",
"changed-deps",
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-08-22 20:50 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-08-22 20:50 UTC (permalink / raw
To: gentoo-commits
commit: 6b17633d0d74cb8fa0eeb46c8ee4f4f05d0e38b2
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Tue Aug 22 20:43:25 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Tue Aug 22 20:49:55 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=6b17633d
eclean-pkg: add --clean-invalid to help, fix parsing
* Rename --clean-invalids -> --clean-invalid
* Add --clean-invalid to help
* Make it default-off for now to match the option name (rather than
the help saying --no-clean-invalids).
(I could go either way on what the actual default should be, but this is more
conservative for now.)
* Fix typo in option name.
* Make it a package specific arg, not a global one.
Bug: https://bugs.gentoo.org/900224
Fixes: 204b1abca2c0cbbdad188c7fd4b84452c2f630a0
Fixes: a16d0d4fbfb4614832c4b682b41284a9050af29f
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index c8381a0..aa8b20f 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -261,6 +261,11 @@ def printUsage(_error=None, help=None, unresolved_invalids=None):
+ " - delete packages for which ebuild dependencies have changed",
file=out,
)
+ print(
+ yellow(" --clean-invalid")
+ + " - cleanup invalid binpkgs",
+ file=out,
+ )
print(
yellow(" -i, --ignore-failure")
+ " - ignore failure to locate PKGDIR",
@@ -413,8 +418,8 @@ def parseArgs(options={}):
options["ignore-failure"] = True
elif o in ("-u", "--unique-use"):
options["unique-use"] = True
- elif o in ("-N", "--skip-invalids"):
- options["clean-invalids"] = False
+ elif o in ("-N", "--skip-invalid"):
+ options["clean-invalid"] = False
else:
return_code = False
# sanity check of --deep only options:
@@ -435,7 +440,7 @@ def parseArgs(options={}):
# here are the different allowed command line options (getopt args)
getopt_options = {"short": {}, "long": {}}
- getopt_options["short"]["global"] = "CdDipqe:t:nhVvN"
+ getopt_options["short"]["global"] = "CdDipqe:t:nhVv"
getopt_options["long"]["global"] = [
"nocolor",
"deep",
@@ -450,14 +455,14 @@ def parseArgs(options={}):
"help",
"version",
"verbose",
- "clean-inavlids",
]
getopt_options["short"]["distfiles"] = "fs:"
getopt_options["long"]["distfiles"] = ["fetch-restricted", "size-limit="]
- getopt_options["short"]["packages"] = "i"
+ getopt_options["short"]["packages"] = "iN"
getopt_options["long"]["packages"] = [
"ignore-failure",
"changed-deps",
+ "clean-invalid",
"unique-use",
]
# set default options, except 'nocolor', which is set in main()
@@ -474,8 +479,8 @@ def parseArgs(options={}):
options["verbose"] = False
options["changed-deps"] = False
options["ignore-failure"] = False
+ options["clean-invalid"] = False
options["unique-use"] = False
- options["clean-invalids"] = True
# if called by a well-named symlink, set the action accordingly:
action = None
# temp print line to ensure it is the svn/branch code running, etc..
@@ -621,7 +626,7 @@ def doAction(action, options, exclude={}, output=None):
output.set_colors("deprecated")
output.list_pkgs(deprecated)
if action in ["packages"]:
- if invalids and options["clean-invalids"]:
+ if invalids and options["clean-invalid"]:
if type(invalids) == list:
printUsage(_error="invalid_paths", unresolved_invalids=invalids)
sys.exit(1)
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-08-22 20:35 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-08-22 20:35 UTC (permalink / raw
To: gentoo-commits
commit: 204b1abca2c0cbbdad188c7fd4b84452c2f630a0
Author: Siddhanth Rathod <xsiddhanthrathod <AT> gmail <DOT> com>
AuthorDate: Tue Aug 22 13:55:48 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Tue Aug 22 20:34:30 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=204b1abc
eclean-pkg: Limit invalid search to 'packages' action
Bug: https://bugs.gentoo.org/900224
Fixes: a16d0d4fbfb4614832c4b682b41284a9050af29f
Signed-off-by: Siddhanth Rathod <xsiddhanthrathod <AT> gmail.com>
Closes: https://github.com/gentoo/gentoolkit/pull/30
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 46 +++++++++++++++++++++++---------------------
1 file changed, 24 insertions(+), 22 deletions(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index 40fc4f9..c8381a0 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -435,7 +435,7 @@ def parseArgs(options={}):
# here are the different allowed command line options (getopt args)
getopt_options = {"short": {}, "long": {}}
- getopt_options["short"]["global"] = "CdDipqe:t:nhVv"
+ getopt_options["short"]["global"] = "CdDipqe:t:nhVvN"
getopt_options["long"]["global"] = [
"nocolor",
"deep",
@@ -450,6 +450,7 @@ def parseArgs(options={}):
"help",
"version",
"verbose",
+ "clean-inavlids",
]
getopt_options["short"]["distfiles"] = "fs:"
getopt_options["long"]["distfiles"] = ["fetch-restricted", "size-limit="]
@@ -619,28 +620,29 @@ def doAction(action, options, exclude={}, output=None):
)
output.set_colors("deprecated")
output.list_pkgs(deprecated)
- if invalids and options["clean-invalids"]:
- if type(invalids) == list:
- printUsage(_error="invalid_paths", unresolved_invalids=invalids)
- sys.exit(1)
- verb = "were"
- if options["pretend"]:
- verb = "would be"
- if not options["quiet"]:
- print()
- print(
- (
- pp.emph(" The following ")
- + red("invalid")
- + pp.emph(" binpkgs were found")
+ if action in ["packages"]:
+ if invalids and options["clean-invalids"]:
+ if type(invalids) == list:
+ printUsage(_error="invalid_paths", unresolved_invalids=invalids)
+ sys.exit(1)
+ verb = "were"
+ if options["pretend"]:
+ verb = "would be"
+ if not options["quiet"]:
+ print()
+ print(
+ (
+ pp.emph(" The following ")
+ + red("invalid")
+ + pp.emph(" binpkgs were found")
+ )
)
- )
- output.set_colors("invalid")
- output.list_pkgs(invalids)
- clean_size = cleaner.clean_pkgs(invalids, pkgdir)
- output.total("invalid", clean_size, len(invalids), verb, action)
- else:
- cleaner.clean_pkgs(invalids, pkgdir)
+ output.set_colors("invalid")
+ output.list_pkgs(invalids)
+ clean_size = cleaner.clean_pkgs(invalids, pkgdir)
+ output.total("invalid", clean_size, len(invalids), verb, action)
+ else:
+ cleaner.clean_pkgs(invalids, pkgdir)
def main():
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-08-21 5:16 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-08-21 5:16 UTC (permalink / raw
To: gentoo-commits
commit: a16d0d4fbfb4614832c4b682b41284a9050af29f
Author: Siddhanth Rathod <xsiddhanthrathod <AT> gmail <DOT> com>
AuthorDate: Thu Mar 23 12:14:36 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Mon Aug 21 05:16:18 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=a16d0d4f
eclean-pkg: deal with invalid binpkgs
Changes required in portage -> https://github.com/gentoo/portage/pull/1016
Bug: https://bugs.gentoo.org/900224
Signed-off-by: Siddhanth Rathod <xsiddhanthrathod <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 47 +++++++++++++++++++++++++++++++++++++----
pym/gentoolkit/eclean/output.py | 14 ++++++++++++
pym/gentoolkit/eclean/search.py | 9 ++++++--
3 files changed, 64 insertions(+), 6 deletions(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index fa94a55..40fc4f9 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -13,7 +13,6 @@ __version__ = "git"
__productname__ = "eclean"
__description__ = "A cleaning tool for Gentoo distfiles and binaries."
-
import os
import sys
import re
@@ -21,7 +20,7 @@ import time
import getopt
import portage
-from portage.output import white, yellow, turquoise, green
+from portage.output import white, yellow, turquoise, green, red
import gentoolkit.pprinter as pp
from gentoolkit.eclean.search import (
@@ -47,7 +46,7 @@ def printVersion():
print("Distributed under the terms of the GNU General Public License v2")
-def printUsage(_error=None, help=None):
+def printUsage(_error=None, help=None, unresolved_invalids=None):
"""Print help message. May also print partial help to stderr if an
error from {'options','actions'} is specified."""
@@ -63,10 +62,25 @@ def printUsage(_error=None, help=None):
"merged-distfiles-options",
"time",
"size",
+ "invalid_paths",
):
_error = None
if not _error and not help:
help = "all"
+ if _error == "invalid_paths":
+ print(
+ pp.error(
+ "eclean was not able to remove invalid binpkgs due to missing features in the currently installed portage"
+ ),
+ file=out,
+ )
+ print(
+ pp.error("Please remove the following binpkgs manually:"),
+ file=out,
+ )
+ for invalid in unresolved_invalids:
+ print(pp.error(invalid), file=out)
+ return
if _error == "time":
print(pp.error("Wrong time specification"), file=out)
print(
@@ -399,6 +413,8 @@ def parseArgs(options={}):
options["ignore-failure"] = True
elif o in ("-u", "--unique-use"):
options["unique-use"] = True
+ elif o in ("-N", "--skip-invalids"):
+ options["clean-invalids"] = False
else:
return_code = False
# sanity check of --deep only options:
@@ -458,6 +474,7 @@ def parseArgs(options={}):
options["changed-deps"] = False
options["ignore-failure"] = False
options["unique-use"] = False
+ options["clean-invalids"] = True
# if called by a well-named symlink, set the action accordingly:
action = None
# temp print line to ensure it is the svn/branch code running, etc..
@@ -527,7 +544,7 @@ def doAction(action, options, exclude={}, output=None):
if not options["quiet"]:
output.einfo("Building file list for " + action + " cleaning...")
if action == "packages":
- clean_me = findPackages(
+ clean_me, invalids = findPackages(
options,
exclude=exclude,
destructive=options["destructive"],
@@ -602,6 +619,28 @@ def doAction(action, options, exclude={}, output=None):
)
output.set_colors("deprecated")
output.list_pkgs(deprecated)
+ if invalids and options["clean-invalids"]:
+ if type(invalids) == list:
+ printUsage(_error="invalid_paths", unresolved_invalids=invalids)
+ sys.exit(1)
+ verb = "were"
+ if options["pretend"]:
+ verb = "would be"
+ if not options["quiet"]:
+ print()
+ print(
+ (
+ pp.emph(" The following ")
+ + red("invalid")
+ + pp.emph(" binpkgs were found")
+ )
+ )
+ output.set_colors("invalid")
+ output.list_pkgs(invalids)
+ clean_size = cleaner.clean_pkgs(invalids, pkgdir)
+ output.total("invalid", clean_size, len(invalids), verb, action)
+ else:
+ cleaner.clean_pkgs(invalids, pkgdir)
def main():
diff --git a/pym/gentoolkit/eclean/output.py b/pym/gentoolkit/eclean/output.py
index 62777b7..ebba499 100644
--- a/pym/gentoolkit/eclean/output.py
+++ b/pym/gentoolkit/eclean/output.py
@@ -39,6 +39,9 @@ class OutputControl:
self.pkg_color = cpv # green
self.numbers = number # turquoise
self.brace = blue
+ if mode == "invalid":
+ self.pkg_color = red
+ self.numbers = teal
elif mode == "deprecated":
self.pkg_color = yellow
self.numbers = teal # darkgreen
@@ -169,6 +172,17 @@ class OutputControl:
)
print(" ===========")
print(self.prettySize(size, True, red), message)
+ elif mode == "invalid":
+ message = (
+ red(str(num_files))
+ + " invalid binpkgs "
+ + verb
+ + " removed from the "
+ + action
+ + " directory"
+ )
+ print(" ===========")
+ print(self.prettySize(size, True, red), message)
elif mode == "deprecated":
message = (
"Total space from "
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index eb26ac8..a2ac0ce 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -517,9 +517,10 @@ def findPackages(
pkgdir: str = None,
port_dbapi=portage.db[portage.root]["porttree"].dbapi,
var_dbapi=portage.db[portage.root]["vartree"].dbapi,
-) -> dict[str, list[str]]:
+) -> tuple[dict[str, list[str]], dict[str, list[str]]]:
"""Find obsolete binary packages.
+ @param invalid_paths:
@param options: dict of options determined at runtime
@type options: dict
@param exclude: exclusion dict (as defined in the exclude.parseExcludeFile class)
@@ -638,8 +639,12 @@ def findPackages(
binpkg_path = bin_dbapi.bintree.getname(cpv)
dead_binpkgs.setdefault(cpv, []).append(binpkg_path)
+ try:
+ invalid_paths = bin_dbapi.bintree.invalid_paths
+ except AttributeError:
+ invalid_paths = bin_dbapi.bintree.invalids
- return dead_binpkgs
+ return dead_binpkgs, invalid_paths
# vim: set ts=4 sw=4 tw=79:
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2023-08-21 5:16 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2023-08-21 5:16 UTC (permalink / raw
To: gentoo-commits
commit: d93602634628ee5f0063b14efa52f98b141d2cc9
Author: Siddhanth Rathod <xsiddhanthrathod <AT> gmail <DOT> com>
AuthorDate: Thu Apr 6 15:17:28 2023 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Mon Aug 21 05:16:18 2023 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=d9360263
eclean: Add type hints
Signed-off-by: Siddhanth Rathod <xsiddhanthrathod <AT> gmail.com>
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 3288208..eb26ac8 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -8,6 +8,7 @@ import os
import stat
import sys
from functools import partial
+from typing import Optional
import portage
from portage.dep import Atom, use_reduce
@@ -508,15 +509,15 @@ def _deps_equal(deps_a, eapi_a, deps_b, eapi_b, uselist=None):
def findPackages(
- options,
- exclude=None,
- destructive=False,
- time_limit=0,
- package_names=False,
- pkgdir=None,
+ options: dict[str, bool],
+ exclude: Optional[dict] = None,
+ destructive: bool = False,
+ time_limit: Optional[int] = 0,
+ package_names: Optional[bool] = False,
+ pkgdir: str = None,
port_dbapi=portage.db[portage.root]["porttree"].dbapi,
var_dbapi=portage.db[portage.root]["vartree"].dbapi,
-):
+) -> dict[str, list[str]]:
"""Find obsolete binary packages.
@param options: dict of options determined at runtime
@@ -563,7 +564,7 @@ def findPackages(
# Dictionary of binary packages to clean. Organized as cpv->[pkgs] in order
# to support FEATURES=binpkg-multi-instance.
- dead_binpkgs = {}
+ dead_binpkgs: dict[str, list[str]] = {}
keep_binpkgs = {}
bin_dbapi = portage.binarytree(pkgdir=pkgdir, settings=var_dbapi.settings).dbapi
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2022-07-11 7:17 Brian Dolbec
0 siblings, 0 replies; 43+ messages in thread
From: Brian Dolbec @ 2022-07-11 7:17 UTC (permalink / raw
To: gentoo-commits
commit: bf3eb16e451fd1bdee8ef03a0d22e0040e033f19
Author: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
AuthorDate: Mon Jul 11 06:41:33 2022 +0000
Commit: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
CommitDate: Mon Jul 11 06:53:33 2022 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=bf3eb16e
eclean/pkgindex.py: Fix typo in function call
File "/usr/lib/python3.10/site-packages/gentoolkit/eclean/pkgindex.py", line
60, in clean_pkgs_index
if self.get_emaint_binhost():
AttributeError: 'PkgIndex' object has no attribute 'get_emaint_binhost'. Did
you mean: '_get_emaint_binhost'?
Also fix too many parameters in line 68 for the self.controller() call
Bug: https://bugs.gentoo.org/857555
Signed-off-by: Brian Dolbec <dolsen <AT> gentoo.org>
pym/gentoolkit/eclean/pkgindex.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/pym/gentoolkit/eclean/pkgindex.py b/pym/gentoolkit/eclean/pkgindex.py
index d0878a1..7d6fade 100644
--- a/pym/gentoolkit/eclean/pkgindex.py
+++ b/pym/gentoolkit/eclean/pkgindex.py
@@ -57,7 +57,7 @@ class PkgIndex:
statinfo = os.stat(file_)
size1 = statinfo.st_size
show_progress = not quiet
- if self.get_emaint_binhost():
+ if self._get_emaint_binhost():
self.taskmaster = TaskHandler(show_progress_bar=show_progress)
tasks = [self.binhost]
self.taskmaster.run_tasks(tasks)
@@ -65,7 +65,7 @@ class PkgIndex:
self.call_emaint()
statinfo = os.stat(file_)
clean_size = size1 - statinfo.st_size
- self.controller("\n", clean_size, "Packages Index", file_, "Index")
+ self.controller(clean_size, "Packages Index", file_, "Index")
return clean_size
def call_emaint(self):
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2022-07-09 22:45 Brian Dolbec
0 siblings, 0 replies; 43+ messages in thread
From: Brian Dolbec @ 2022-07-09 22:45 UTC (permalink / raw
To: gentoo-commits
commit: 125142667c338eddb3777433e513fbe58e7bfd45
Author: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
AuthorDate: Sat Jul 9 22:35:33 2022 +0000
Commit: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
CommitDate: Sat Jul 9 22:42:48 2022 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=12514266
eclean: add early return for empty DISTDIR
Gentoo-bug-url: https://bugs.gentoo.org/703282
Signed-off-by: Brian Dolbec <dolsen <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 365970c..60ced02 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -111,6 +111,10 @@ class DistfilesSearch:
saved = {}
deprecated = {}
installed_included = False
+ # Check if DISTDIR is empty, return early
+ if not os.listdir(_distdir):
+ return clean_me, saved, deprecated
+
# create a big CPV->SRC_URI dict of packages
# whose distfiles should be kept
if (not destructive) or fetch_restricted:
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2022-07-09 21:46 Brian Dolbec
0 siblings, 0 replies; 43+ messages in thread
From: Brian Dolbec @ 2022-07-09 21:46 UTC (permalink / raw
To: gentoo-commits
commit: 680dd160e1e5a41f008bd1a47afa2da2eb25714f
Author: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
AuthorDate: Sat Jul 9 21:41:39 2022 +0000
Commit: Brian Dolbec <dolsen <AT> gentoo <DOT> org>
CommitDate: Sat Jul 9 21:45:04 2022 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=680dd160
eclean: Complete migration to imported emaint code
eclean-pkg now uses imported emaint binhost code to run,
Properly handle --quiet option for progressbar suppression from emaint.
Keeps the standalone emaint binhost subprocess call as backup.
Closes: https://bugs.gentoo.org/688550
Signed-off-by: Brian Dolbec <dolsen <AT> gentoo.org>
pym/gentoolkit/eclean/clean.py | 5 ++--
pym/gentoolkit/eclean/cli.py | 2 +-
pym/gentoolkit/eclean/pkgindex.py | 56 ++++++++++++++++++---------------------
3 files changed, 30 insertions(+), 33 deletions(-)
diff --git a/pym/gentoolkit/eclean/clean.py b/pym/gentoolkit/eclean/clean.py
index a6358a4..37a042a 100644
--- a/pym/gentoolkit/eclean/clean.py
+++ b/pym/gentoolkit/eclean/clean.py
@@ -19,8 +19,9 @@ class CleanUp:
or bypassing/ignoring
"""
- def __init__(self, controller):
+ def __init__(self, controller, quiet):
self.controller = controller
+ self.quiet = quiet
def clean_dist(self, clean_dict):
"""Calculate size of each entry for display, prompt user if needed,
@@ -64,7 +65,7 @@ class CleanUp:
# emaint is not yet importable so call it
# print a blank line here for separation
print()
- clean_size += index_control.call_emaint()
+ clean_size += index_control.clean_pkgs_index(self.quiet)
# return total size of deleted or to delete files
return clean_size
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index e3bc21a..23c7f3b 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -555,7 +555,7 @@ def doAction(action, options, exclude={}, output=None):
)
# initialize our cleaner
- cleaner = CleanUp(output.progress_controller)
+ cleaner = CleanUp(output.progress_controller, options["quiet"])
# actually clean files if something was found
if clean_me:
diff --git a/pym/gentoolkit/eclean/pkgindex.py b/pym/gentoolkit/eclean/pkgindex.py
index 617b437..d0878a1 100644
--- a/pym/gentoolkit/eclean/pkgindex.py
+++ b/pym/gentoolkit/eclean/pkgindex.py
@@ -11,6 +11,11 @@ import gentoolkit.pprinter as pp
from gentoolkit.eprefix import EPREFIX
import portage
+from portage.module import (
+ InvalidModuleName,
+ Modules,
+)
+from portage.emaint.main import TaskHandler
class PkgIndex:
@@ -32,37 +37,36 @@ class PkgIndex:
@sets: self.binhost to BinhostHandler class
@rtype: boolean
"""
- # About noqa below: I don't understand how this code can run at all.
- # TODO: verify soundness
try:
- self.emaint_control = Modules() # noqa
+ self.emaint_control = Modules()
self.binhost = self.emaint_control._get_class("binhost")
- except InvalidModuleName as er: # noqa
+ except InvalidModuleName as er:
print(pp.error("Error importing emaint binhost module"), file=sys.stderr)
print(pp.error("Original error: " + er), file=sys.stderr)
except:
return False
return True
- def _load_modules(self):
- """Import the emaint modules and report the success/fail of them"""
- try:
- from emaint.module import Modules # noqa
- from emaint.main import TaskHandler # noqa
- except ImportError:
- return False
- return True
+ def clean_pkgs_index(self, quiet):
+ """This will clean the binpkgs packages index file
- def clean_pkgs_index(
- self,
- ):
- """This will clean the binpkgs packages index file"""
- go = self._load_modules()
- if go:
- if self.get_emaint_binhost():
- self.taskmaster = TaskHandler(show_progress_bar=True) # noqa
- tasks = [self.binhost]
- self.taskmaster.run_tasks(tasks)
+ @param quiet: boolean
+ @return: the difference in file size
+ """
+ file_ = os.path.join(portage.settings["PKGDIR"], "Packages")
+ statinfo = os.stat(file_)
+ size1 = statinfo.st_size
+ show_progress = not quiet
+ if self.get_emaint_binhost():
+ self.taskmaster = TaskHandler(show_progress_bar=show_progress)
+ tasks = [self.binhost]
+ self.taskmaster.run_tasks(tasks)
+ else:
+ self.call_emaint()
+ statinfo = os.stat(file_)
+ clean_size = size1 - statinfo.st_size
+ self.controller("\n", clean_size, "Packages Index", file_, "Index")
+ return clean_size
def call_emaint(self):
"""Run the stand alone emaint script from
@@ -71,9 +75,6 @@ class PkgIndex:
@rtype: integer
@return: the difference in file size
"""
- file_ = os.path.join(portage.settings["PKGDIR"], "Packages")
- statinfo = os.stat(file_)
- size1 = statinfo.st_size
try:
retcode = subprocess.call(self.emaint_cmd, shell=True)
if retcode < 0:
@@ -83,8 +84,3 @@ class PkgIndex:
)
except OSError as e:
print(pp.error("Execution failed:" + e), file=sys.stderr)
- print()
- statinfo = os.stat(file_)
- clean_size = size1 - statinfo.st_size
- self.controller(clean_size, "Packages Index", file_, "Index")
- return clean_size
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2022-06-01 10:25 Yixun Lan
0 siblings, 0 replies; 43+ messages in thread
From: Yixun Lan @ 2022-06-01 10:25 UTC (permalink / raw
To: gentoo-commits
commit: b3251b763901ec5a5d7690fd8ec2885629104ee2
Author: Yixun Lan <dlan <AT> gentoo <DOT> org>
AuthorDate: Wed Jun 1 10:23:49 2022 +0000
Commit: Yixun Lan <dlan <AT> gentoo <DOT> org>
CommitDate: Wed Jun 1 10:24:41 2022 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=b3251b76
fix missing short option "-u" for --unique-use
Signed-off-by: Yixun Lan <dlan <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index c129d5e..e3bc21a 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -397,7 +397,7 @@ def parseArgs(options={}):
options["changed-deps"] = True
elif o in ("-i", "--ignore-failure"):
options["ignore-failure"] = True
- elif o in ("--unique-use"):
+ elif o in ("-u", "--unique-use"):
options["unique-use"] = True
else:
return_code = False
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2022-05-28 15:28 Yixun Lan
0 siblings, 0 replies; 43+ messages in thread
From: Yixun Lan @ 2022-05-28 15:28 UTC (permalink / raw
To: gentoo-commits
commit: 2fffbd450df2443bbd671f7ec760051ce3c930e9
Author: Yixun Lan <dlan <AT> gentoo <DOT> org>
AuthorDate: Wed May 25 08:08:09 2022 +0000
Commit: Yixun Lan <dlan <AT> gentoo <DOT> org>
CommitDate: Sat May 28 15:24:32 2022 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=2fffbd45
implement --unique-use option for eclean-pkg
this will help to clean duplicated binpkg which has exact same USEs flags enabled,
and it will choose the more recent version according to BUILD_TIME by default
Closes: https://bugs.gentoo.org/727576
Closes: https://github.com/gentoo/gentoolkit/pull/20
Signed-off-by: Yixun Lan <dlan <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 14 +++++++++++++-
pym/gentoolkit/eclean/search.py | 26 ++++++++++++++++++++++++++
2 files changed, 39 insertions(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index 2ad2ae9..c129d5e 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -252,6 +252,11 @@ def printUsage(_error=None, help=None):
+ " - ignore failure to locate PKGDIR",
file=out,
)
+ print(
+ yellow(" -u, --unique-use")
+ + " - keep unique packages which have no duplicated USE",
+ file=out,
+ )
print(file=out)
if _error in ("distfiles-options", "merged-distfiles-options") or help in (
"all",
@@ -392,6 +397,8 @@ def parseArgs(options={}):
options["changed-deps"] = True
elif o in ("-i", "--ignore-failure"):
options["ignore-failure"] = True
+ elif o in ("--unique-use"):
+ options["unique-use"] = True
else:
return_code = False
# sanity check of --deep only options:
@@ -431,7 +438,11 @@ def parseArgs(options={}):
getopt_options["short"]["distfiles"] = "fs:"
getopt_options["long"]["distfiles"] = ["fetch-restricted", "size-limit="]
getopt_options["short"]["packages"] = "i"
- getopt_options["long"]["packages"] = ["ignore-failure", "changed-deps"]
+ getopt_options["long"]["packages"] = [
+ "ignore-failure",
+ "changed-deps",
+ "unique-use",
+ ]
# set default options, except 'nocolor', which is set in main()
options["interactive"] = False
options["pretend"] = False
@@ -446,6 +457,7 @@ def parseArgs(options={}):
options["verbose"] = False
options["changed-deps"] = False
options["ignore-failure"] = False
+ options["unique-use"] = False
# if called by a well-named symlink, set the action accordingly:
action = None
# temp print line to ensure it is the svn/branch code running, etc..
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index cb695c0..365970c 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -560,6 +560,7 @@ def findPackages(
# Dictionary of binary packages to clean. Organized as cpv->[pkgs] in order
# to support FEATURES=binpkg-multi-instance.
dead_binpkgs = {}
+ keep_binpkgs = {}
bin_dbapi = portage.binarytree(pkgdir=pkgdir, settings=var_dbapi.settings).dbapi
for cpv in bin_dbapi.cpv_all():
@@ -575,6 +576,28 @@ def findPackages(
if mtime >= time_limit:
continue
+ # Exclude if binpkg has exact same USEs
+ if not destructive and options["unique-use"]:
+ keys = ("CPV", "EAPI", "USE")
+ binpkg_metadata = dict(zip(keys, bin_dbapi.aux_get(cpv, keys)))
+ cpv_key = "_".join(binpkg_metadata[key] for key in keys)
+ if cpv_key in keep_binpkgs:
+ old_cpv = keep_binpkgs[cpv_key]
+ # compare BUILD_TIME, keep the new one
+ old_time = int(bin_dbapi.aux_get(old_cpv, ["BUILD_TIME"])[0])
+ new_time = int(bin_dbapi.aux_get(cpv, ["BUILD_TIME"])[0])
+ drop_cpv = old_cpv if new_time >= old_time else cpv
+
+ binpkg_path = bin_dbapi.bintree.getname(drop_cpv)
+ dead_binpkgs.setdefault(drop_cpv, []).append(binpkg_path)
+
+ if new_time >= old_time:
+ keep_binpkgs[cpv_key] = cpv
+ else:
+ continue
+ else:
+ keep_binpkgs[cpv_key] = cpv
+
# Exclude if binpkg exists in the porttree and not --deep
if not destructive and port_dbapi.cpv_exists(cpv):
if not options["changed-deps"]:
@@ -605,6 +628,9 @@ def findPackages(
if buildtime == bin_dbapi.aux_get(cpv, ["BUILD_TIME"])[0]:
continue
+ if not destructive and options["unique-use"]:
+ del keep_binpkgs[cpv_key]
+
binpkg_path = bin_dbapi.bintree.getname(cpv)
dead_binpkgs.setdefault(cpv, []).append(binpkg_path)
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2022-02-09 10:48 Sam James
0 siblings, 0 replies; 43+ messages in thread
From: Sam James @ 2022-02-09 10:48 UTC (permalink / raw
To: gentoo-commits
commit: 76c188b5189375b3b3b7c92f31491990e29a1669
Author: Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Wed Feb 9 10:47:54 2022 +0000
Commit: Sam James <sam <AT> gentoo <DOT> org>
CommitDate: Wed Feb 9 10:48:13 2022 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=76c188b5
*/*: reformat with latest Black 22.1.0
Signed-off-by: Sam James <sam <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index bb6deeb..2ad2ae9 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -309,7 +309,7 @@ def parseSize(size):
@raise ParseArgsException: in case of failure
"""
- units = {"G": (1024 ** 3), "M": (1024 ** 2), "K": 1024, "B": 1}
+ units = {"G": (1024**3), "M": (1024**2), "K": 1024, "B": 1}
try:
match = re.match(r"^(?P<value>\d+)(?P<unit>[GMKBgmkb])?$", size)
size = int(match.group("value"))
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2020-06-09 17:29 Zac Medico
0 siblings, 0 replies; 43+ messages in thread
From: Zac Medico @ 2020-06-09 17:29 UTC (permalink / raw
To: gentoo-commits
commit: c0b355cf51ca73ad9dd52a0f5c1aebf807be42fc
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Tue Jun 9 01:54:46 2020 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Jun 9 01:57:07 2020 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=c0b355cf
eclean --changed-deps: fix EAPI logic (bug 727596)
Use separate variables for binary package and corresponding ebuild
EAPI values, since the ebuild EAPI may have changed. This avoids
a possible InvalidAtom exception as reported in bug 727596.
Bug: https://bugs.gentoo.org/727596
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 51edc63..f68aacf 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -490,12 +490,12 @@ class DistfilesSearch(object):
return clean_me, saved
-def _deps_equal(deps_a, deps_b, eapi, uselist=None):
+def _deps_equal(deps_a, eapi_a, deps_b, eapi_b, uselist=None):
"""Compare two dependency lists given a set of USE flags"""
if deps_a == deps_b: return True
- deps_a = use_reduce(deps_a, uselist=uselist, eapi=eapi, token_class=Atom)
- deps_b = use_reduce(deps_b, uselist=uselist, eapi=eapi, token_class=Atom)
+ deps_a = use_reduce(deps_a, uselist=uselist, eapi=eapi_a, token_class=Atom)
+ deps_b = use_reduce(deps_b, uselist=uselist, eapi=eapi_b, token_class=Atom)
strip_slots(deps_a)
strip_slots(deps_b)
return deps_a == deps_b
@@ -578,12 +578,14 @@ def findPackages(
if not options['changed-deps']:
continue
- keys = ('RDEPEND', 'PDEPEND')
- binpkg_deps = ' '.join(bin_dbapi.aux_get(cpv, keys))
- ebuild_deps = ' '.join(port_dbapi.aux_get(cpv, keys))
- uselist = bin_dbapi.aux_get(cpv, ['USE'])[0].split()
+ dep_keys = ('RDEPEND', 'PDEPEND')
+ keys = ('EAPI', 'USE') + dep_keys
+ binpkg_metadata = dict(zip(keys, bin_dbapi.aux_get(cpv, keys)))
+ ebuild_metadata = dict(zip(keys, port_dbapi.aux_get(cpv, keys)))
- if _deps_equal(binpkg_deps, ebuild_deps, cpv.eapi, uselist):
+ if _deps_equal(' '.join(binpkg_metadata[key] for key in dep_keys), binpkg_metadata['EAPI'],
+ ' '.join(ebuild_metadata[key] for key in dep_keys), ebuild_metadata['EAPI'],
+ frozenset(binpkg_metadata['USE'].split())):
continue
if destructive and var_dbapi.cpv_exists(cpv):
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2020-03-12 16:51 Matt Turner
0 siblings, 0 replies; 43+ messages in thread
From: Matt Turner @ 2020-03-12 16:51 UTC (permalink / raw
To: gentoo-commits
commit: 98cdccea8f0abc88d360c0c7143263b210910611
Author: Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue Feb 18 21:39:08 2020 +0000
Commit: Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed Mar 11 23:33:01 2020 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=98cdccea
eclean: Add option to delete binpkgs with changed deps
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 7 ++++++-
pym/gentoolkit/eclean/search.py | 24 +++++++++++++++++++++++-
2 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index 1a99b3e..39aafd3 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -147,6 +147,8 @@ def printUsage(_error=None, help=None):
or help in ('all','packages'):
print( "Available", yellow("options"),"for the",
green("packages"),"action:", file=out)
+ print( yellow(" --changed-deps")+
+ " - delete packages for which ebuild dependencies have changed", file=out)
print( yellow(" -i, --ignore-failure")+
" - ignore failure to locate PKGDIR", file=out)
print( file=out)
@@ -263,6 +265,8 @@ def parseArgs(options={}):
options['size-limit'] = parseSize(a)
elif o in ("-v", "--verbose") and not options['quiet']:
options['verbose'] = True
+ elif o in ("--changed-deps"):
+ options['changed-deps'] = True
elif o in ("-i", "--ignore-failure"):
options['ignore-failure'] = True
else:
@@ -290,7 +294,7 @@ def parseArgs(options={}):
getopt_options['short']['distfiles'] = "fs:"
getopt_options['long']['distfiles'] = ["fetch-restricted", "size-limit="]
getopt_options['short']['packages'] = "i"
- getopt_options['long']['packages'] = ["ignore-failure"]
+ getopt_options['long']['packages'] = ["ignore-failure", "changed-deps"]
# set default options, except 'nocolor', which is set in main()
options['interactive'] = False
options['pretend'] = False
@@ -303,6 +307,7 @@ def parseArgs(options={}):
options['fetch-restricted'] = False
options['size-limit'] = 0
options['verbose'] = False
+ options['changed-deps'] = False
options['ignore-failure'] = False
# if called by a well-named symlink, set the action accordingly:
action = None
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 0efefdb..51edc63 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -13,6 +13,8 @@ import sys
from functools import partial
import portage
+from portage.dep import Atom, use_reduce
+from portage.dep._slot_operator import strip_slots
import gentoolkit.pprinter as pp
from gentoolkit.eclean.exclude import (exclDictMatchCP, exclDictExpand,
@@ -488,6 +490,17 @@ class DistfilesSearch(object):
return clean_me, saved
+def _deps_equal(deps_a, deps_b, eapi, uselist=None):
+ """Compare two dependency lists given a set of USE flags"""
+ if deps_a == deps_b: return True
+
+ deps_a = use_reduce(deps_a, uselist=uselist, eapi=eapi, token_class=Atom)
+ deps_b = use_reduce(deps_b, uselist=uselist, eapi=eapi, token_class=Atom)
+ strip_slots(deps_a)
+ strip_slots(deps_b)
+ return deps_a == deps_b
+
+
def findPackages(
options,
exclude=None,
@@ -562,7 +575,16 @@ def findPackages(
# Exclude if binpkg exists in the porttree and not --deep
if not destructive and port_dbapi.cpv_exists(cpv):
- continue
+ if not options['changed-deps']:
+ continue
+
+ keys = ('RDEPEND', 'PDEPEND')
+ binpkg_deps = ' '.join(bin_dbapi.aux_get(cpv, keys))
+ ebuild_deps = ' '.join(port_dbapi.aux_get(cpv, keys))
+ uselist = bin_dbapi.aux_get(cpv, ['USE'])[0].split()
+
+ if _deps_equal(binpkg_deps, ebuild_deps, cpv.eapi, uselist):
+ continue
if destructive and var_dbapi.cpv_exists(cpv):
# Exclude if an instance of the package is installed due to
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2020-03-12 16:51 Matt Turner
0 siblings, 0 replies; 43+ messages in thread
From: Matt Turner @ 2020-03-12 16:51 UTC (permalink / raw
To: gentoo-commits
commit: e4f75fd355cfee672f64a1f9c42a19894e9f7703
Author: Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue Feb 18 21:05:00 2020 +0000
Commit: Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Sat Mar 7 06:07:38 2020 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=e4f75fd3
eclean: Rewrite findPackages()
I found the original code to be nearly incomprehensible. Instead of
populating a dict of potential binpkgs to remove and then removing from
the to-be-removed list, just selectively add to-be-removed packages.
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 113 +++++++++++++++++++---------------------
1 file changed, 55 insertions(+), 58 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 58bd97e..0efefdb 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -498,89 +498,86 @@ def findPackages(
port_dbapi=portage.db[portage.root]["porttree"].dbapi,
var_dbapi=portage.db[portage.root]["vartree"].dbapi
):
- """Find all obsolete binary packages.
-
- XXX: packages are found only by symlinks.
- Maybe i should also return .tbz2 files from All/ that have
- no corresponding symlinks.
+ """Find obsolete binary packages.
@param options: dict of options determined at runtime
- @param exclude: an exclusion dict as defined in
- exclude.parseExcludeFile class.
- @param destructive: boolean, defaults to False
- @param time_limit: integer time value as returned by parseTime()
- @param package_names: boolean, defaults to False.
- used only if destructive=True
- @param pkgdir: path to the binary package dir being checked
+ @type options: dict
+ @param exclude: exclusion dict (as defined in the exclude.parseExcludeFile class)
+ @type exclude: dict, optional
+ @param destructive: binpkg is obsolete if not installed (default: `False`)
+ @type destructive: bool, optional
+ @param time_limit: exclude binpkg if newer than time value as returned by parseTime()
+ @type time_limit: int, optional
+ @param package_names: exclude all binpkg versions if package is installed
+ (used with `destructive=True`) (default: `False`)
+ @type package_names: bool, optional
+ @param pkgdir: path to the binpkg cache (PKGDIR)
+ @type pkgdir: str
@param port_dbapi: defaults to portage.db[portage.root]["porttree"].dbapi
- can be overridden for tests.
- @param var_dbapi: defaults to portage.db[portage.root]["vartree"].dbapi
- can be overridden for tests.
+ Can be overridden for tests.
+ @param var_dbapi: defaults to portage.db[portage.root]["vartree"].dbapi
+ Can be overridden for tests.
+ @return binary packages to remove. e.g. {'cat/pkg-ver': [filepath]}
@rtype: dict
- @return clean_me i.e. {'cat/pkg-ver.tbz2': [filepath],}
"""
if exclude is None:
exclude = {}
- clean_me = {}
- # create a full package dictionary
- # now do an access test, os.walk does not error for "no read permission"
+ # Access test, os.walk does not error for "no read permission"
try:
test = os.listdir(pkgdir)
del test
except EnvironmentError as er:
if options['ignore-failure']:
exit(0)
- print( pp.error("Error accessing PKGDIR." ), file=sys.stderr)
- print( pp.error("(Check your make.conf file and environment)."), file=sys.stderr)
- print( pp.error("Error: %s" %str(er)), file=sys.stderr)
+ print(pp.error("Error accessing PKGDIR."), file=sys.stderr)
+ print(pp.error("(Check your make.conf file and environment)."), file=sys.stderr)
+ print(pp.error("Error: %s" % str(er)), file=sys.stderr)
exit(1)
- # if portage supports FEATURES=binpkg-multi-instance, then
- # cpv_all can return multiple instances per cpv, where
- # instances are distinguishable by some extra attributes
- # provided by portage's _pkg_str class
+ # Create a dictionary of all installed packages
+ if destructive and package_names:
+ installed = dict.fromkeys(var_dbapi.cp_all())
+ else:
+ installed = {}
+
+ # Dictionary of binary packages to clean. Organized as cpv->[pkgs] in order
+ # to support FEATURES=binpkg-multi-instance.
+ dead_binpkgs = {}
+
bin_dbapi = portage.binarytree(pkgdir=pkgdir, settings=var_dbapi.settings).dbapi
for cpv in bin_dbapi.cpv_all():
- mtime = int(bin_dbapi.aux_get(cpv, ['_mtime_'])[0])
- if time_limit and mtime >= time_limit:
- # time-limit exclusion
- continue
- # dict is cpv->[pkgs] (supports binpkg-multi-instance)
- clean_me.setdefault(cpv, []).append(cpv)
+ cp = portage.cpv_getkey(cpv)
- # keep only obsolete ones
- if destructive and package_names:
- cp_all = dict.fromkeys(var_dbapi.cp_all())
- else:
- cp_all = {}
- for cpv in list(clean_me):
- if exclDictMatchCP(exclude,portage.cpv_getkey(cpv)):
- # exclusion because of the exclude file
- del clean_me[cpv]
+ # Exclude per --exclude-file=...
+ if exclDictMatchCP(exclude, cp):
continue
+
+ # Exclude if binpkg is newer than --time-limit=...
+ if time_limit:
+ mtime = int(bin_dbapi.aux_get(cpv, ['_mtime_'])[0])
+ if mtime >= time_limit:
+ continue
+
+ # Exclude if binpkg exists in the porttree and not --deep
if not destructive and port_dbapi.cpv_exists(cpv):
- # exclusion because pkg still exists (in porttree)
- del clean_me[cpv]
continue
+
if destructive and var_dbapi.cpv_exists(cpv):
+ # Exclude if an instance of the package is installed due to
+ # the --package-names option.
+ if cp in installed and port_dbapi.cpv_exists(cpv):
+ continue
+
+ # Exclude if BUILD_TIME of binpkg is same as vartree
buildtime = var_dbapi.aux_get(cpv, ['BUILD_TIME'])[0]
- clean_me[cpv] = [pkg for pkg in clean_me[cpv]
- # only keep path if BUILD_TIME is identical with vartree
- if bin_dbapi.aux_get(pkg, ['BUILD_TIME'])[0] != buildtime]
- if not clean_me[cpv]:
- # nothing we can clean for this package
- del clean_me[cpv]
+ if buildtime == bin_dbapi.aux_get(cpv, ['BUILD_TIME'])[0]:
continue
- if portage.cpv_getkey(cpv) in cp_all and port_dbapi.cpv_exists(cpv):
- # exclusion because of --package-names
- del clean_me[cpv]
- # the getname method correctly supports FEATURES=binpkg-multi-instance,
- # allowing for multiple paths per cpv (the API used here is also compatible
- # with older portage which does not support binpkg-multi-instance)
- for cpv, pkgs in clean_me.items():
- clean_me[cpv] = [bin_dbapi.bintree.getname(pkg) for pkg in pkgs]
+ binpkg_path = bin_dbapi.bintree.getname(cpv)
+ dead_binpkgs.setdefault(cpv, []).append(binpkg_path)
+
+ return dead_binpkgs
- return clean_me
+# vim: set ts=4 sw=4 tw=79:
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2020-01-03 4:31 Matt Turner
0 siblings, 0 replies; 43+ messages in thread
From: Matt Turner @ 2020-01-03 4:31 UTC (permalink / raw
To: gentoo-commits
commit: fba1039db46156089840896d3b6aa90076872d73
Author: Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue Dec 10 18:29:27 2019 +0000
Commit: Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue Dec 10 18:42:44 2019 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=fba1039d
eclean: Fix typos
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>
pym/gentoolkit/eclean/cli.py | 4 ++--
pym/gentoolkit/eclean/search.py | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index 1d2f52b..1a99b3e 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -304,7 +304,7 @@ def parseArgs(options={}):
options['size-limit'] = 0
options['verbose'] = False
options['ignore-failure'] = False
- # if called by a well-named symlink, set the acction accordingly:
+ # if called by a well-named symlink, set the action accordingly:
action = None
# temp print line to ensure it is the svn/branch code running, etc..
#print( "###### svn/branch/gentoolkit_eclean ####### ==> ", os.path.basename(sys.argv[0]))
@@ -400,7 +400,7 @@ def doAction(action,options,exclude={}, output=None):
)
# initialize our cleaner
- cleaner = CleanUp( output.progress_controller)
+ cleaner = CleanUp(output.progress_controller)
# actually clean files if something was found
if clean_me:
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index ce455a3..58bd97e 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -574,7 +574,7 @@ def findPackages(
del clean_me[cpv]
continue
if portage.cpv_getkey(cpv) in cp_all and port_dbapi.cpv_exists(cpv):
- # exlusion because of --package-names
+ # exclusion because of --package-names
del clean_me[cpv]
# the getname method correctly supports FEATURES=binpkg-multi-instance,
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2019-12-05 16:51 Matt Turner
0 siblings, 0 replies; 43+ messages in thread
From: Matt Turner @ 2019-12-05 16:51 UTC (permalink / raw
To: gentoo-commits
commit: 3f25e307ba5116892beb48fec2809500882dcf4f
Author: Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon Dec 2 22:15:14 2019 +0000
Commit: Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Thu Dec 5 02:23:20 2019 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=3f25e307
eclean: Delete empty directories
Closes: https://bugs.gentoo.org/671592
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>
pym/gentoolkit/eclean/clean.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/pym/gentoolkit/eclean/clean.py b/pym/gentoolkit/eclean/clean.py
index e00bcc0..c62abb9 100644
--- a/pym/gentoolkit/eclean/clean.py
+++ b/pym/gentoolkit/eclean/clean.py
@@ -139,6 +139,10 @@ class CleanUp(object):
# only count size if successfully deleted and not a link
if statinfo.st_nlink == 1:
clean_size += statinfo.st_size
+ try:
+ os.rmdir(os.path.dirname(file_))
+ except OSError:
+ pass
except EnvironmentError as er:
print( pp.error("Could not delete "+file_), file=sys.stderr)
print( pp.error("Error: %s" %str(er)), file=sys.stderr)
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2019-12-05 16:51 Matt Turner
0 siblings, 0 replies; 43+ messages in thread
From: Matt Turner @ 2019-12-05 16:51 UTC (permalink / raw
To: gentoo-commits
commit: af21727e35249aceda111b345b50c9dbf95de805
Author: Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon Dec 2 20:28:15 2019 +0000
Commit: Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Thu Dec 5 02:23:20 2019 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=af21727e
eclean: Inline _sort_keys method
The boilerplate for calling this method was larger than what it actually
contained. Additionally I think this change will allow the loop to run
on a generator rather than a full list.
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>
pym/gentoolkit/eclean/clean.py | 21 ++++++---------------
1 file changed, 6 insertions(+), 15 deletions(-)
diff --git a/pym/gentoolkit/eclean/clean.py b/pym/gentoolkit/eclean/clean.py
index fd59976..e00bcc0 100644
--- a/pym/gentoolkit/eclean/clean.py
+++ b/pym/gentoolkit/eclean/clean.py
@@ -36,10 +36,9 @@ class CleanUp(object):
@return: total size that was cleaned
"""
file_type = 'file'
- clean_keys = self._sort_keys(clean_dict)
clean_size = 0
- # clean all entries one by one
- for key in clean_keys:
+ # clean all entries one by one; sorting helps reading
+ for key in sorted(clean_dict):
clean_size += self._clean_files(clean_dict[key], key, file_type)
# return total size of deleted or to delete files
return clean_size
@@ -57,10 +56,9 @@ class CleanUp(object):
@return: total size that was cleaned
"""
file_type = 'binary package'
- clean_keys = self._sort_keys(clean_dict)
clean_size = 0
- # clean all entries one by one
- for key in clean_keys:
+ # clean all entries one by one; sorting helps reading
+ for key in sorted(clean_dict):
clean_size += self._clean_files(clean_dict[key], key, file_type)
# run 'emaint --fix' here
@@ -83,10 +81,9 @@ class CleanUp(object):
@return: total size that would be cleaned
"""
file_type = 'file'
- clean_keys = self._sort_keys(clean_dict)
clean_size = 0
- # tally all entries one by one
- for key in clean_keys:
+ # tally all entries one by one; sorting helps reading
+ for key in sorted(clean_dict):
key_size = self._get_size(clean_dict[key])
self.controller(key_size, key, clean_dict[key], file_type)
clean_size += key_size
@@ -110,12 +107,6 @@ class CleanUp(object):
print( pp.error("Error: %s" %str(er)), file=sys.stderr)
return key_size
- def _sort_keys(self, clean_dict):
- """Returns a list of sorted dictionary keys."""
- # sorting helps reading
- clean_keys = sorted(clean_dict)
- return clean_keys
-
def _clean_files(self, files, key, file_type):
"""File removal function."""
clean_size = 0
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2019-09-13 23:34 Zac Medico
0 siblings, 0 replies; 43+ messages in thread
From: Zac Medico @ 2019-09-13 23:34 UTC (permalink / raw
To: gentoo-commits
commit: d03253b11ba067c6a396af76fa18ed4d9c27d4ea
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Fri Sep 13 23:33:15 2019 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Sep 13 23:33:54 2019 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=d03253b1
eclean: tox fix: local variable 'e' is assigned to but never used
Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
pym/gentoolkit/eclean/pkgindex.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/pkgindex.py b/pym/gentoolkit/eclean/pkgindex.py
index 584a5bc..c1f5f9d 100644
--- a/pym/gentoolkit/eclean/pkgindex.py
+++ b/pym/gentoolkit/eclean/pkgindex.py
@@ -55,7 +55,7 @@ class PkgIndex(object):
try:
from emaint.module import Modules # noqa
from emaint.main import TaskHandler # noqa
- except ImportError as e:
+ except ImportError:
return False
return True
^ permalink raw reply related [flat|nested] 43+ messages in thread
* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2017-12-12 5:51 Zac Medico
0 siblings, 0 replies; 43+ messages in thread
From: Zac Medico @ 2017-12-12 5:51 UTC (permalink / raw
To: gentoo-commits
commit: 8eda4c13bee9c6a78b2489deec210c8b37375029
Author: Göktürk Yüksek <gokturk <AT> gentoo <DOT> org>
AuthorDate: Mon Dec 11 14:21:59 2017 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Tue Dec 12 05:51:19 2017 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=8eda4c13
eclean: add the missing ')' in the help string of --package-names
pym/gentoolkit/eclean/cli.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index d8c5bd2..7ffb596 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -120,7 +120,7 @@ def printUsage(_error=None, help=None):
print( yellow(" -i, --interactive")+
" - ask confirmation before deletions", file=out)
print( yellow(" -n, --package-names")+
- " - protect all versions (when --deep", file=out)
+ " - protect all versions (when --deep)", file=out)
print( yellow(" -p, --pretend")+
" - only display what would be cleaned", file=out)
print( yellow(" -q, --quiet")+
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2016-07-03 6:29 Zac Medico
0 siblings, 0 replies; 43+ messages in thread
From: Zac Medico @ 2016-07-03 6:29 UTC (permalink / raw
To: gentoo-commits
commit: a6b0f09806845ba305eb1a39b90ff8752552c9df
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sun Jul 3 06:27:43 2016 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Sun Jul 3 06:29:17 2016 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=a6b0f098
eclean: fix DeprecationWarning: Passing nonexistent key 'NOCOLOR'
pym/gentoolkit/eclean/cli.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index aa9af3f..d8c5bd2 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -1,6 +1,6 @@
#!/usr/bin/python
-# Copyright 2003-2010 Gentoo Foundation
+# Copyright 2003-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
@@ -448,7 +448,7 @@ def main():
"""Parse command line and execute all actions."""
# set default options
options = {}
- options['nocolor'] = (port_settings["NOCOLOR"] in ('yes','true')
+ options['nocolor'] = (port_settings.get("NOCOLOR") in ('yes','true')
or not sys.stdout.isatty())
if options['nocolor']:
pp.output.nocolor()
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2016-07-01 6:19 Zac Medico
0 siblings, 0 replies; 43+ messages in thread
From: Zac Medico @ 2016-07-01 6:19 UTC (permalink / raw
To: gentoo-commits
commit: 1952ecb9f2912968dd2f4487089fa50ed3d3bad1
Author: Zac Medico <zmedico <AT> gentoo <DOT> org>
AuthorDate: Sat Jun 25 23:55:33 2016 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Jul 1 06:18:27 2016 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=1952ecb9
eclean: optimize binpkg-multi-instance support from bug 586658
Use the portage binarytree API to optimize binary package access,
so that metadata is read from $PKGDIR/Packages instead of from the
individual binary packages. Symlinks will now be ignored, since
portage hasn't used symlinks for years, and there's no harm
in ignoring them now. The APIs used are compatible with very old
portage, though they internally support binpkg-multi-instance
in recent versions of portage.
X-Gentoo-bug: 586658
X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=586658
pym/gentoolkit/eclean/search.py | 47 +++++++++++++++++++----------------------
1 file changed, 22 insertions(+), 25 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index ce796f5..b9d111a 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -538,29 +538,20 @@ def findPackages(
print( pp.error("(Check your make.conf file and environment)."), file=sys.stderr)
print( pp.error("Error: %s" %str(er)), file=sys.stderr)
exit(1)
- for root, dirs, files in os.walk(pkgdir):
- if root[-3:] == 'All':
+
+ # if portage supports FEATURES=binpkg-multi-instance, then
+ # cpv_all can return multiple instances per cpv, where
+ # instances are distinguishable by some extra attributes
+ # provided by portage's _pkg_str class
+ bin_dbapi = portage.binarytree(pkgdir=pkgdir, settings=var_dbapi.settings).dbapi
+ for cpv in bin_dbapi.cpv_all():
+ mtime = int(bin_dbapi.aux_get(cpv, ['_mtime_'])[0])
+ if time_limit and mtime >= time_limit:
+ # time-limit exclusion
continue
- for file in files:
- if file[-5:] == ".tbz2":
- category = os.path.basename(root)
- cpv = category+"/"+file[:-5]
- elif file[-5:] == ".xpak":
- category = os.path.basename(os.path.dirname(root))
- cpv = category+"/"+file.rpartition('-')[0]
- else:
- # ignore other files
- continue
- path = os.path.join(root, file)
- st = os.lstat(path)
- if time_limit and (st[stat.ST_MTIME] >= time_limit):
- # time-limit exclusion
- continue
- # dict is cpv->[files] (2 files in general, because of symlink)
- clean_me.setdefault(cpv,[]).append(path)
- #if os.path.islink(path):
- if stat.S_ISLNK(st[stat.ST_MODE]):
- clean_me[cpv].append(os.path.realpath(path))
+ # dict is cpv->[pkgs] (supports binpkg-multi-instance)
+ clean_me.setdefault(cpv, []).append(cpv)
+
# keep only obsolete ones
if destructive and package_names:
cp_all = dict.fromkeys(var_dbapi.cp_all())
@@ -576,10 +567,10 @@ def findPackages(
del clean_me[cpv]
continue
if destructive and var_dbapi.cpv_exists(cpv):
- buildtime = var_dbapi.aux_get(cpv, ['BUILD_TIME'])[0].encode('utf-8').strip()
- clean_me[cpv] = [path for path in clean_me[cpv]
+ buildtime = var_dbapi.aux_get(cpv, ['BUILD_TIME'])[0]
+ clean_me[cpv] = [pkg for pkg in clean_me[cpv]
# only keep path if BUILD_TIME is identical with vartree
- if portage.xpak.tbz2(path).getfile('BUILD_TIME').strip() != buildtime]
+ if bin_dbapi.aux_get(pkg, ['BUILD_TIME'])[0] != buildtime]
if not clean_me[cpv]:
# nothing we can clean for this package
del clean_me[cpv]
@@ -588,4 +579,10 @@ def findPackages(
# exlusion because of --package-names
del clean_me[cpv]
+ # the getname method correctly supports FEATURES=binpkg-multi-instance,
+ # allowing for multiple paths per cpv (the API used here is also compatible
+ # with older portage which does not support binpkg-multi-instance)
+ for cpv, pkgs in clean_me.items():
+ clean_me[cpv] = [bin_dbapi.bintree.getname(pkg) for pkg in pkgs]
+
return clean_me
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2016-06-24 21:40 Zac Medico
0 siblings, 0 replies; 43+ messages in thread
From: Zac Medico @ 2016-06-24 21:40 UTC (permalink / raw
To: gentoo-commits
commit: 824953dd70d650ee0b2c057b0dfb44efb8f56a9b
Author: Manuel Mommertz <2kmm <AT> gmx <DOT> de>
AuthorDate: Fri Jun 24 15:40:56 2016 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Jun 24 21:40:28 2016 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=824953dd
eclean: Keep only packages with BUILD_TIME equal to installed one
X-Gentoo-bug: 586658
X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=586658
Acked-by: Paul Varner <fuzzyray <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 22 +++++++++++++---------
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 7b261b8..ce796f5 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -562,24 +562,28 @@ def findPackages(
if stat.S_ISLNK(st[stat.ST_MODE]):
clean_me[cpv].append(os.path.realpath(path))
# keep only obsolete ones
- if destructive:
- dbapi = var_dbapi
- if package_names:
- cp_all = dict.fromkeys(dbapi.cp_all())
- else:
- cp_all = {}
+ if destructive and package_names:
+ cp_all = dict.fromkeys(var_dbapi.cp_all())
else:
- dbapi = port_dbapi
cp_all = {}
for cpv in list(clean_me):
if exclDictMatchCP(exclude,portage.cpv_getkey(cpv)):
# exclusion because of the exclude file
del clean_me[cpv]
continue
- if dbapi.cpv_exists(cpv):
- # exclusion because pkg still exists (in porttree or vartree)
+ if not destructive and port_dbapi.cpv_exists(cpv):
+ # exclusion because pkg still exists (in porttree)
del clean_me[cpv]
continue
+ if destructive and var_dbapi.cpv_exists(cpv):
+ buildtime = var_dbapi.aux_get(cpv, ['BUILD_TIME'])[0].encode('utf-8').strip()
+ clean_me[cpv] = [path for path in clean_me[cpv]
+ # only keep path if BUILD_TIME is identical with vartree
+ if portage.xpak.tbz2(path).getfile('BUILD_TIME').strip() != buildtime]
+ if not clean_me[cpv]:
+ # nothing we can clean for this package
+ del clean_me[cpv]
+ continue
if portage.cpv_getkey(cpv) in cp_all and port_dbapi.cpv_exists(cpv):
# exlusion because of --package-names
del clean_me[cpv]
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2016-06-24 21:36 Zac Medico
0 siblings, 0 replies; 43+ messages in thread
From: Zac Medico @ 2016-06-24 21:36 UTC (permalink / raw
To: gentoo-commits
commit: 78a446d0859fd1d3c2922d821fe0664099287312
Author: Manuel Mommertz <2kmm <AT> gmx <DOT> de>
AuthorDate: Fri Jun 24 15:40:56 2016 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Jun 24 21:35:15 2016 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=78a446d0
eclean: Keep only packages with BUILD_TIME equal to installed one
X-Gentoo-bug: 586658
X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=586658
Acked-by: Paul Varner <fuzzyray <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 22 +++++++++++++---------
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 7b261b8..ce796f5 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -562,24 +562,28 @@ def findPackages(
if stat.S_ISLNK(st[stat.ST_MODE]):
clean_me[cpv].append(os.path.realpath(path))
# keep only obsolete ones
- if destructive:
- dbapi = var_dbapi
- if package_names:
- cp_all = dict.fromkeys(dbapi.cp_all())
- else:
- cp_all = {}
+ if destructive and package_names:
+ cp_all = dict.fromkeys(var_dbapi.cp_all())
else:
- dbapi = port_dbapi
cp_all = {}
for cpv in list(clean_me):
if exclDictMatchCP(exclude,portage.cpv_getkey(cpv)):
# exclusion because of the exclude file
del clean_me[cpv]
continue
- if dbapi.cpv_exists(cpv):
- # exclusion because pkg still exists (in porttree or vartree)
+ if not destructive and port_dbapi.cpv_exists(cpv):
+ # exclusion because pkg still exists (in porttree)
del clean_me[cpv]
continue
+ if destructive and var_dbapi.cpv_exists(cpv):
+ buildtime = var_dbapi.aux_get(cpv, ['BUILD_TIME'])[0].encode('utf-8').strip()
+ clean_me[cpv] = [path for path in clean_me[cpv]
+ # only keep path if BUILD_TIME is identical with vartree
+ if portage.xpak.tbz2(path).getfile('BUILD_TIME').strip() != buildtime]
+ if not clean_me[cpv]:
+ # nothing we can clean for this package
+ del clean_me[cpv]
+ continue
if portage.cpv_getkey(cpv) in cp_all and port_dbapi.cpv_exists(cpv):
# exlusion because of --package-names
del clean_me[cpv]
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2016-06-23 20:01 Paul Varner
0 siblings, 0 replies; 43+ messages in thread
From: Paul Varner @ 2016-06-23 20:01 UTC (permalink / raw
To: gentoo-commits
commit: 0c95b6b4aea0b7baa86c6f41e6b643c477f60203
Author: Paul Varner <fuzzyray <AT> gentoo <DOT> org>
AuthorDate: Thu Jun 23 20:00:15 2016 +0000
Commit: Paul Varner <fuzzyray <AT> gentoo <DOT> org>
CommitDate: Thu Jun 23 20:00:15 2016 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=0c95b6b4
eclean: Apply handle binpkgs with .xpak suffix patch from bug 586658
Author: Manuel Mommertz <2kmm <AT> gmx.de>
X-Gentoo-bug: 586658
X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=586658
pym/gentoolkit/eclean/search.py | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index f771ffc..7b261b8 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -542,18 +542,22 @@ def findPackages(
if root[-3:] == 'All':
continue
for file in files:
- if not file[-5:] == ".tbz2":
- # ignore non-tbz2 files
+ if file[-5:] == ".tbz2":
+ category = os.path.basename(root)
+ cpv = category+"/"+file[:-5]
+ elif file[-5:] == ".xpak":
+ category = os.path.basename(os.path.dirname(root))
+ cpv = category+"/"+file.rpartition('-')[0]
+ else:
+ # ignore other files
continue
path = os.path.join(root, file)
- category = os.path.split(root)[-1]
- cpv = category+"/"+file[:-5]
st = os.lstat(path)
if time_limit and (st[stat.ST_MTIME] >= time_limit):
# time-limit exclusion
continue
# dict is cpv->[files] (2 files in general, because of symlink)
- clean_me[cpv] = [path]
+ clean_me.setdefault(cpv,[]).append(path)
#if os.path.islink(path):
if stat.S_ISLNK(st[stat.ST_MODE]):
clean_me[cpv].append(os.path.realpath(path))
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2015-10-19 21:33 Paul Varner
0 siblings, 0 replies; 43+ messages in thread
From: Paul Varner @ 2015-10-19 21:33 UTC (permalink / raw
To: gentoo-commits
commit: 00fa83ca375064247acf92c402e7920c1a5c922f
Author: Paul Varner <fuzzyray <AT> gentoo <DOT> org>
AuthorDate: Mon Oct 19 21:32:28 2015 +0000
Commit: Paul Varner <fuzzyray <AT> gentoo <DOT> org>
CommitDate: Mon Oct 19 21:32:28 2015 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=00fa83ca
eclean: Fix traceback error caused by adding -i option
pym/gentoolkit/eclean/cli.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
index 1c8e66f..aa9af3f 100644
--- a/pym/gentoolkit/eclean/cli.py
+++ b/pym/gentoolkit/eclean/cli.py
@@ -303,6 +303,7 @@ def parseArgs(options={}):
options['fetch-restricted'] = False
options['size-limit'] = 0
options['verbose'] = False
+ options['ignore-failure'] = False
# if called by a well-named symlink, set the acction accordingly:
action = None
# temp print line to ensure it is the svn/branch code running, etc..
^ permalink raw reply related [flat|nested] 43+ messages in thread* [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/
@ 2015-10-15 21:39 Paul Varner
0 siblings, 0 replies; 43+ messages in thread
From: Paul Varner @ 2015-10-15 21:39 UTC (permalink / raw
To: gentoo-commits
commit: 68283db9d0a775c334087a27601edf042987294c
Author: Marc Joliet <marcec <AT> gmx <DOT> de>
AuthorDate: Thu Oct 15 21:16:57 2015 +0000
Commit: Paul Varner <fuzzyray <AT> gentoo <DOT> org>
CommitDate: Thu Oct 15 21:38:45 2015 +0000
URL: https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=68283db9
eclean: fix eclean-pkg -n -d Bug #472020
Signed-off-by: Paul Varner <fuzzyray <AT> gentoo.org>
pym/gentoolkit/eclean/search.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
index 5fcb6bc..1a83fd4 100644
--- a/pym/gentoolkit/eclean/search.py
+++ b/pym/gentoolkit/eclean/search.py
@@ -574,7 +574,7 @@ def findPackages(
# exclusion because pkg still exists (in porttree or vartree)
del clean_me[cpv]
continue
- if portage.cpv_getkey(cpv) in cp_all:
+ if portage.cpv_getkey(cpv) in cp_all and port_dbapi.cpv_exists(cpv):
# exlusion because of --package-names
del clean_me[cpv]
^ permalink raw reply related [flat|nested] 43+ messages in thread
end of thread, other threads:[~2024-05-03 5:42 UTC | newest]
Thread overview: 43+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-05-03 5:42 [gentoo-commits] proj/gentoolkit:master commit in: pym/gentoolkit/eclean/ Sam James
-- strict thread matches above, loose matches on Subject: below --
2024-05-03 5:40 Sam James
2024-05-03 5:40 Sam James
2024-05-03 5:40 Sam James
2024-05-03 5:11 Sam James
2024-02-18 2:19 Brian Dolbec
2024-02-18 2:19 Brian Dolbec
2024-01-19 7:03 Sam James
2024-01-19 6:15 Sam James
2024-01-12 19:16 Matt Turner
2023-12-03 7:51 Sam James
2023-10-08 3:28 Sam James
2023-10-08 3:28 Sam James
2023-09-29 7:20 Sam James
2023-08-24 19:52 Sam James
2023-08-22 20:54 Sam James
2023-08-22 20:52 Sam James
2023-08-22 20:50 Sam James
2023-08-22 20:50 Sam James
2023-08-22 20:35 Sam James
2023-08-21 5:16 Sam James
2023-08-21 5:16 Sam James
2022-07-11 7:17 Brian Dolbec
2022-07-09 22:45 Brian Dolbec
2022-07-09 21:46 Brian Dolbec
2022-06-01 10:25 Yixun Lan
2022-05-28 15:28 Yixun Lan
2022-02-09 10:48 Sam James
2020-06-09 17:29 Zac Medico
2020-03-12 16:51 Matt Turner
2020-03-12 16:51 Matt Turner
2020-01-03 4:31 Matt Turner
2019-12-05 16:51 Matt Turner
2019-12-05 16:51 Matt Turner
2019-09-13 23:34 Zac Medico
2017-12-12 5:51 Zac Medico
2016-07-03 6:29 Zac Medico
2016-07-01 6:19 Zac Medico
2016-06-24 21:40 Zac Medico
2016-06-24 21:36 Zac Medico
2016-06-23 20:01 Paul Varner
2015-10-19 21:33 Paul Varner
2015-10-15 21:39 Paul Varner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox