[gentoo-commits] proj/pkgcore/pkgcore:master commit in: src/pkgcore/sync/

[gentoo-commits] proj/pkgcore/pkgcore:master commit in: src/pkgcore/sync/

[Arthur Zamarin]

commit:     67f57cab42a865339096d0305f7773006dafab2e
Author:     Brian Harring <ferringb <AT> gmail <DOT> com>
AuthorDate: Tue Jan 10 10:00:23 2023 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Tue Jan 17 20:46:03 2023 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcore.git/commit/?id=67f57cab

fix(sync): Add loggger.debug() of the sync command actually being invoked.

Additionally, since we know the sync subsystem was written before basic sanity
of the frontend, and we know that it invokes processes that write to stdout/stderr,
force a flush of those handles to ensure that we don't wind up with interlaced output
from two processes.

The need to bury a flush here is a sign that the abstraction needs redesign, but
that's a problem for a later date.

Signed-off-by: Brian Harring <ferringb <AT> gmail.com>
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcore/sync/base.py | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/src/pkgcore/sync/base.py b/src/pkgcore/sync/base.py
index 1114532fb..f015cd2c0 100644
--- a/src/pkgcore/sync/base.py
+++ b/src/pkgcore/sync/base.py
@@ -14,6 +14,7 @@ __all__ = (
 import os
 import pwd
 import stat
+import sys
 import typing
 from importlib import import_module
 
@@ -22,6 +23,7 @@ from snakeoil import process
 from .. import os_data
 from ..config.hint import ConfigHint, configurable
 from ..exceptions import PkgcoreUserException
+from ..log import logger
 
 
 class SyncError(PkgcoreUserException):
@@ -195,6 +197,12 @@ class ExternalSyncer(Syncer):
         # Note: stderr is explicitly forced to stdout since that's how it was originally done.
         # This can be changed w/ a discussion.
         kwargs.setdefault("fd_pipes", {1: 1, 2: 1})
+        logger.debug("sync invoking command %r, kwargs %r", command, kwargs)
+        # since we're intermixing two processes writing to stdout/stderr- us, and what we're invoking-
+        # force a flush to keep output from being interlaced.  This is not hugely optimal, but
+        # the CLI/observability integration needs refactoring anyways.
+        sys.stdout.flush()
+        sys.stderr.flush()
         return process.spawn.spawn(
             command, uid=self.uid, gid=self.gid, env=self.env, **kwargs
         )

[gentoo-commits] proj/pkgcore/pkgcore:master commit in: src/pkgcore/sync/

[Arthur Zamarin]

commit:     51d850a4e60b71d49da4102203c7a622c73156f1
Author:     Brian Harring <ferringb <AT> gmail <DOT> com>
AuthorDate: Tue Jan 10 09:36:48 2023 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Fri Jan 20 08:01:15 2023 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcore.git/commit/?id=51d850a4

refactor(sync): Remove unused variable assignment.

We're only trying to limit the retries, thus use islice
instead.

Signed-off-by: Brian Harring <ferringb <AT> gmail.com>
Closes: https://github.com/pkgcore/pkgcore/pull/394
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcore/sync/rsync.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/pkgcore/sync/rsync.py b/src/pkgcore/sync/rsync.py
index b0f032731..acf1793bf 100644
--- a/src/pkgcore/sync/rsync.py
+++ b/src/pkgcore/sync/rsync.py
@@ -7,6 +7,7 @@ import os
 import socket
 import tempfile
 import time
+from itertools import islice
 
 from snakeoil.osutils import pjoin
 
@@ -142,7 +143,7 @@ class rsync_syncer(base.ExternalSyncer):
 
         # zip limits to the shortest iterable
         ret = None
-        for count, ip in zip(range(self.retries), self._get_ips()):
+        for ip in islice(self._get_ips(), self.retries):
             cmd = [
                 self.binary_path,
                 self.uri.replace(self.hostname, ip, 1),

[gentoo-commits] proj/pkgcore/pkgcore:master commit in: src/pkgcore/sync/

[Arthur Zamarin]

commit:     c75f522b4c9ddf46ba9e5c72f8a4cdb3132b54cd
Author:     Vitaly Zdanevich <zdanevich.vitaly <AT> ya <DOT> ru>
AuthorDate: Fri Mar  8 13:10:52 2024 +0000
Commit:     Arthur Zamarin <arthurzam <AT> gentoo <DOT> org>
CommitDate: Sat Mar  9 18:04:54 2024 +0000
URL:        https://gitweb.gentoo.org/proj/pkgcore/pkgcore.git/commit/?id=c75f522b

Add git@

Signed-off-by: Vitaly Zdanevich <zdanevich.vitaly <AT> ya.ru>
Closes: https://github.com/pkgcore/pkgcore/pull/434
Signed-off-by: Arthur Zamarin <arthurzam <AT> gentoo.org>

 src/pkgcore/sync/git.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/pkgcore/sync/git.py b/src/pkgcore/sync/git.py
index 48d6b3518..ce8f6f8e7 100644
--- a/src/pkgcore/sync/git.py
+++ b/src/pkgcore/sync/git.py
@@ -11,9 +11,10 @@ class git_syncer(base.VcsSyncer):
     supported_uris = (
         ("git://", 5),
         ("git+", 5),
+        ("git@", 5),
     )
 
-    supported_protocols = ("http://", "https://", "git://")
+    supported_protocols = ("http://", "https://", "git://", "git@")
     supported_exts = (".git",)
 
     @classmethod