[gentoo-commits] proj/pkgcore/pkgcheck:master commit in: src/pkgcheck/addons/

[gentoo-commits] proj/pkgcore/pkgcheck:master commit in: src/pkgcheck/addons/

[Arthur Zamarin]

commit:     a19030fda246d49a3ff24f44a0f005e99dd65a1a
Author:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
AuthorDate: Tue Oct  3 18:25:27 2023 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Tue Oct  3 18:25:27 2023 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcheck.git/commit/?id=a19030fd

addons.git: add helping message on failure on git remote setup

Upon initial failure to get the git diff-tree output (so not the hot
flow, but sad failure flow), try to catch output of incorrectly
configured git remote (the remote should have a correct HEAD configured).

Recommend the user to run `git remote set-head origin -a` to solve it.

Resolves: https://github.com/pkgcore/pkgcheck/issues/608
Resolves: https://github.com/pkgcore/pkgdev/issues/107
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcheck/addons/git.py | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/src/pkgcheck/addons/git.py b/src/pkgcheck/addons/git.py
index 5d41ba3b..0db52091 100644
--- a/src/pkgcheck/addons/git.py
+++ b/src/pkgcheck/addons/git.py
@@ -353,6 +353,26 @@ class _ScanGit(argparse.Action):
     def default_ref(self, remote):
         return "HEAD" if self.staged else f"{remote}..HEAD"
 
+    def _try_git_remote(self, parser, namespace):
+        """Try to catch case of missing git remote HEAD ref."""
+        try:
+            subprocess.run(
+                ["git", "rev-parse", namespace.git_remote],
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+                cwd=namespace.target_repo.location,
+                check=True,
+                encoding="utf8",
+            )
+        except FileNotFoundError as exc:
+            parser.error(str(exc))
+        except subprocess.CalledProcessError as exc:
+            error = exc.stderr.splitlines()[0]
+            if "ambiguous argument" in error and "unknown revision" in error:
+                parser.error(
+                    f"failed running git: {error}\nSuggested to configure the remote by running 'git remote set-head {namespace.git_remote} -a'"
+                )
+
     def generate_restrictions(self, parser, namespace, ref):
         """Generate restrictions for a given diff command."""
         try:
@@ -368,6 +388,8 @@ class _ScanGit(argparse.Action):
             parser.error(str(exc))
         except subprocess.CalledProcessError as exc:
             error = exc.stderr.splitlines()[0]
+            if "ambiguous argument" in error and "unknown revision" in error:
+                self._try_git_remote(parser, namespace)
             parser.error(f"failed running git: {error}")
 
         if not p.stdout:

[gentoo-commits] proj/pkgcore/pkgcheck:master commit in: src/pkgcheck/addons/

[Arthur Zamarin]

commit:     9e52de6f3e238d832b86b5abe196e6e3439ce2bd
Author:     Martin Mokry <martin-kokos <AT> users <DOT> noreply <DOT> github <DOT> com>
AuthorDate: Tue Feb  4 10:09:40 2025 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Fri Feb  7 19:26:26 2025 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcheck.git/commit/?id=9e52de6f

git: suggest clearing cache on `git log` error

In some cases, the cache gets corrupted, showing:

pkgcheck scan: error: failed running git log: fatal: Invalid revision range <hash>..origin/HEAD

Recommend clearing the cache with `pkgcheck cache -R` as a workaround.

Signed-off-by: Martin Mokry <martin-kokos <AT> protonmail.com>
Closes: https://github.com/pkgcore/pkgcheck/pull/729
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcheck/addons/git.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/pkgcheck/addons/git.py b/src/pkgcheck/addons/git.py
index 1874e8a6..e3de52c0 100644
--- a/src/pkgcheck/addons/git.py
+++ b/src/pkgcheck/addons/git.py
@@ -126,7 +126,12 @@ class GitLog:
         if not self._running:
             if self.proc.poll() or not line:
                 error = self.proc.stderr.read().decode().strip()
-                raise GitError(f"failed running git log: {error}")
+                if "Invalid revision range" in error:
+                    raise GitError(
+                        f"failed running git log: {error}\nTry clearing the cache: pkgcheck cache -R"
+                    )
+                else:
+                    raise GitError(f"failed running git log: {error}")
             self._running = True
             self.git_config.close()
 

[gentoo-commits] proj/pkgcore/pkgcheck:master commit in: src/pkgcheck/addons/

[Arthur Zamarin]

commit:     796cffb5634d50747c73fe989a3de77031aebea0
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Fri Feb 16 23:25:33 2024 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Fri Mar  1 19:19:14 2024 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcheck.git/commit/?id=796cffb5

git: fix no-copies option

This reverts commit 9103513e26f9f2aeade5b563a49697c0e2665e3e.

I originally added --no-find-copies in e688357bdc5773009bb2e106075d9852f2513f89
to suppress git being too clever which ends up confusing our git integration, as
we're not really interested in copies. I was going to use --no-find-copies-harder
but I figured --no-find-copies sounded better as it was less specific and for
our purposes here, we don't want copies at all, so why not?

But --no-find-copies isn't a boolean option, it's just that until git commit
5825268db1058516d05be03d6a8d8d55eea5a943 ('parse-options: fully disable option abbreviation with PARSE_OPT_KEEP_UNKNOWN'),
it tolerated it and expanded it to --no-find-copies-harder. Oops!

Let's use --no-find-copies-harder as we originally should have, as the only
available option to control the behavior, modulo setting a similarity % threshold
with --find-copies=n.

We're fine to do this as, quoting the git-log docs:
"[...] and options applicable to the git-diff[1] command to control how the changes each commit introduces are shown."

Bug: https://bugs.gentoo.org/924718
Bug: https://github.com/pkgcore/pkgcheck/issues/663
Signed-off-by: Sam James <sam <AT> gentoo.org>
Closes: https://github.com/pkgcore/pkgcheck/pull/664
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcheck/addons/git.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/src/pkgcheck/addons/git.py b/src/pkgcheck/addons/git.py
index 752c9cff..7b2435ac 100644
--- a/src/pkgcheck/addons/git.py
+++ b/src/pkgcheck/addons/git.py
@@ -155,8 +155,7 @@ class _ParseGitRepo:
         cmd = shlex.split(self._git_cmd)
         cmd.append(f"--pretty=tformat:%n{'%n'.join(self._format)}")
         cmd.append(commit_range)
-        # https://bugs.gentoo.org/924718
-        # cmd.extend(("--no-find-copies", "--no-find-copies-harder", "--find-renames"))
+        cmd.extend(("--no-find-copies-harder", "--find-renames"))
 
         self.git_log = GitLog(cmd, self.path)
         # discard the initial newline

[gentoo-commits] proj/pkgcore/pkgcheck:master commit in: src/pkgcheck/addons/

[Arthur Zamarin]

commit:     9103513e26f9f2aeade5b563a49697c0e2665e3e
Author:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
AuthorDate: Fri Feb 16 20:38:46 2024 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Fri Feb 16 20:38:46 2024 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcheck.git/commit/?id=9103513e

git: revert log no copies option

It broke with git-2.43.2, until further upstream work by Sam James, I'll
just revert it.

Bug: https://bugs.gentoo.org/924718
Reverts: e688357bdc5773009bb2e106075d9852f2513f89
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcheck/addons/git.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/pkgcheck/addons/git.py b/src/pkgcheck/addons/git.py
index 0db52091..752c9cff 100644
--- a/src/pkgcheck/addons/git.py
+++ b/src/pkgcheck/addons/git.py
@@ -155,7 +155,8 @@ class _ParseGitRepo:
         cmd = shlex.split(self._git_cmd)
         cmd.append(f"--pretty=tformat:%n{'%n'.join(self._format)}")
         cmd.append(commit_range)
-        cmd.extend(("--no-find-copies", "--no-find-copies-harder", "--find-renames"))
+        # https://bugs.gentoo.org/924718
+        # cmd.extend(("--no-find-copies", "--no-find-copies-harder", "--find-renames"))
 
         self.git_log = GitLog(cmd, self.path)
         # discard the initial newline

[gentoo-commits] proj/pkgcore/pkgcheck:master commit in: src/pkgcheck/addons/

[Arthur Zamarin]

commit:     043cbef7cd6c6fa2ca73fbcdf6769ea21cfb5950
Author:     Anna “CyberTailor” <cyber <AT> sysrq <DOT> in>
AuthorDate: Thu Feb  1 16:53:50 2024 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Fri Feb  9 17:55:12 2024 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcheck.git/commit/?id=043cbef7

addons.net: suppress urllib3 import warnings

Most notably, this includes NotOpenSSLWarning:
https://github.com/urllib3/urllib3/issues/3020#issuecomment-1785873825

Signed-off-by: Anna “CyberTailor” <cyber <AT> sysrq.in>
Closes: https://github.com/pkgcore/pkgcheck/pull/661
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcheck/addons/net.py | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/src/pkgcheck/addons/net.py b/src/pkgcheck/addons/net.py
index 6db5432d..782d74d1 100644
--- a/src/pkgcheck/addons/net.py
+++ b/src/pkgcheck/addons/net.py
@@ -2,12 +2,14 @@
 
 import logging
 import os
-
-import requests
+import warnings
 
 from ..checks.network import RequestError, SSLError
 
-# suppress all urllib3 log messages
+# suppress all urllib3 log messages and import warnings
+with warnings.catch_warnings():
+    warnings.simplefilter("ignore")
+    import requests
 logging.getLogger("urllib3").propagate = False
 
 

[gentoo-commits] proj/pkgcore/pkgcheck:master commit in: src/pkgcheck/addons/

[Arthur Zamarin]

commit:     e688357bdc5773009bb2e106075d9852f2513f89
Author:     Sam James <sam <AT> gentoo <DOT> org>
AuthorDate: Thu Sep  7 22:54:44 2023 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Fri Sep  8 12:20:25 2023 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcheck.git/commit/?id=e688357b

addons: git: pass --no-find-copies --no-find-copies-harder --find-renames to git log

I currently have a local git hack to allow configuring git to default to
--find-copies-harder because it's *extremely* useful when working on ebuild
repositories (prompted by a discussion with Eli Schwartz).

Unfortunately, this can confuse pkgcheck's git intergration because it'll
call `git log` like:
```
git log --name-status --diff-filter=ARMD -z --pretty=tformat:%n%h%n%ct cc5b3b9f134a070c548faa4e3de17d615497d0b3..origin/HEAD
```
and get nothing back because (I think) git is interpreting some changes as copies
rather than renames or new files.

Explicitly pass options to disable finding copies, even though normally this
isn't necessary, to keep things working.

Signed-off-by: Sam James <sam <AT> gentoo.org>
Closes: https://github.com/pkgcore/pkgcheck/pull/618
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcheck/addons/git.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/pkgcheck/addons/git.py b/src/pkgcheck/addons/git.py
index 669ac262..5d41ba3b 100644
--- a/src/pkgcheck/addons/git.py
+++ b/src/pkgcheck/addons/git.py
@@ -155,6 +155,7 @@ class _ParseGitRepo:
         cmd = shlex.split(self._git_cmd)
         cmd.append(f"--pretty=tformat:%n{'%n'.join(self._format)}")
         cmd.append(commit_range)
+        cmd.extend(("--no-find-copies", "--no-find-copies-harder", "--find-renames"))
 
         self.git_log = GitLog(cmd, self.path)
         # discard the initial newline

[gentoo-commits] proj/pkgcore/pkgcheck:master commit in: src/pkgcheck/addons/

[Arthur Zamarin]

commit:     4d74d90aa601b5caab445a9dd88c649c0b02856f
Author:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
AuthorDate: Tue Jan 24 19:21:25 2023 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Tue Jan 24 19:21:25 2023 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcheck.git/commit/?id=4d74d90a

_GitCommitPkg: fix no attribute live or slot

Resolves: https://github.com/pkgcore/pkgcheck/issues/380
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcheck/addons/git.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/src/pkgcheck/addons/git.py b/src/pkgcheck/addons/git.py
index 9fd016ce..2547bcc5 100644
--- a/src/pkgcheck/addons/git.py
+++ b/src/pkgcheck/addons/git.py
@@ -264,6 +264,12 @@ class GitRepoPkgs(_ParseGitRepo):
 class _GitCommitPkg(cpv.VersionedCPV):
     """Fake packages encapsulating commits parsed from git log."""
 
+    __slots__ = ("commit", "old", "status", "time")
+
+    # set multiple defaults for the fake package
+    live = False
+    slot = "0"
+
     def __init__(self, category, package, status, version, time, commit, old=None):
         super().__init__(category, package, version)
 

[gentoo-commits] proj/pkgcore/pkgcheck:master commit in: src/pkgcheck/addons/

[Arthur Zamarin]

commit:     0e22d6f56b735b5b7449e5e3f800685bb54651d4
Author:     Daniel M. Weeks <dan <AT> danweeks <DOT> net>
AuthorDate: Thu Jan 19 17:12:06 2023 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Thu Jan 19 20:54:28 2023 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcheck.git/commit/?id=0e22d6f5

Use REPO profile base for profile caching

Prior to this change, scanning an overlay would always cause a profile
cache update since nothing repo-specific was actually getting updated.

Signed-off-by: Daniel M. Weeks <dan <AT> danweeks.net>
Closes: https://github.com/pkgcore/pkgcheck/pull/528
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcheck/addons/profiles.py | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/src/pkgcheck/addons/profiles.py b/src/pkgcheck/addons/profiles.py
index 799cd94a..f9a6862e 100644
--- a/src/pkgcheck/addons/profiles.py
+++ b/src/pkgcheck/addons/profiles.py
@@ -267,7 +267,9 @@ class ProfileAddon(caches.CachedAddon):
                     for profile_obj, profile in self.arch_profiles.get(arch, []):
                         files = self.profile_data.get(profile)
                         try:
-                            cached_profile = cached_profiles[profile.base][profile.path]
+                            cached_profile = cached_profiles[repo.config.profiles_base][
+                                profile.path
+                            ]
                             if files != cached_profile["files"]:
                                 # force refresh of outdated cache entry
                                 raise KeyError
@@ -329,8 +331,8 @@ class ProfileAddon(caches.CachedAddon):
                                 # unsupported EAPI or other issue, profile checks will catch this
                                 continue
 
-                            cached_profiles[profile.base]["update"] = True
-                            cached_profiles[profile.base][profile.path] = {
+                            cached_profiles[repo.config.profiles_base]["update"] = True
+                            cached_profiles[repo.config.profiles_base][profile.path] = {
                                 "files": files,
                                 "masks": masks,
                                 "unmasks": unmasks,