[gentoo-dev] git://anongit.gentoo.org is extremely slow

[gentoo-dev] git://anongit.gentoo.org is extremely slow

[Nikos Chantziaras]

Adding overlays with layman is now extremely slow after the overlays 
were moved to anongit.gentoo.org. Cloning anything beginning with 
"git://anongit.gentoo.org/" is capped at 15KB/s, sometimes 10KB/s.

This is unworkably slow.

Is this intentional?

[gentoo-dev] Re: git://anongit.gentoo.org is extremely slow

[Duncan]

Nikos Chantziaras posted on Wed, 22 Apr 2015 17:11:13 +0300 as excerpted:

> Adding overlays with layman is now extremely slow after the overlays
> were moved to anongit.gentoo.org. Cloning anything beginning with
> "git://anongit.gentoo.org/" is capped at 15KB/s, sometimes 10KB/s.
> 
> This is unworkably slow.
> 
> Is this intentional?

Perhaps the slashdot effect of everybody almost at once needing to delete 
and (re)add their overlays in layman, thus forcing a new clone, in 
ordered to kill the sync errors due to the previously configured host 
disappearing?

I had that happen a couple days ago and once I figured out that the old 
host shutdowns was why I was getting the errors, the delete/re-add 
process went reasonably smoothly, but that might have been before the big 
rush.

Or maybe it's simply suffering an attack DDoS?  Infra?

FWIW, once cloned, updates seem to be reasonably normal/fast, at least 
for the two overlays (kde and x11) I have subscribed.  Maybe they're 
limiting cloning speed due to the above, figuring people will get thru it 
eventually, to keep normal updates reasonable?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

On 23 April 2015 at 15:07, Duncan <1i5t5.duncan@cox.net> wrote:

> Perhaps the slashdot effect of everybody almost at once needing to delete
> and (re)add their overlays in layman, thus forcing a new clone,
>

Eh? That seems incredibly wrong for git.

git remote set-url master <newurl>

Would handle the git side.

You can even tweak git to rewrite all urls for you, like:

[url "git://anongit.gentoo.org"]
    insteadOf = git://git.overlays.gentoo.org

Admittedly it may have been hard for layman to be intelligent about this,
but requiring everyone to remove/add simply for a URI change strikes me as
rather odd here.


-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 1716 bytes --]

Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow

[Brian Dolbec]

On Thu, 23 Apr 2015 15:20:15 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> On 23 April 2015 at 15:07, Duncan <1i5t5.duncan@cox.net> wrote:
> 
> > Perhaps the slashdot effect of everybody almost at once needing to
> > delete and (re)add their overlays in layman, thus forcing a new
> > clone,
> >
> 
> Eh? That seems incredibly wrong for git.
> 
> git remote set-url master <newurl>
> 
> Would handle the git side.
> 
> You can even tweak git to rewrite all urls for you, like:
> 
> [url "git://anongit.gentoo.org"]
>     insteadOf = git://git.overlays.gentoo.org
> 
> Admittedly it may have been hard for layman to be intelligent about
> this, but requiring everyone to remove/add simply for a URI change
> strikes me as rather odd here.
> 
> 

No, layman does try to update the url first, failing that it
deletes/re-adds it.

But I think what happened for some, was, the urls had not yet been
updated or updated correctly in repositories.xml.  That resulted in
failures to update to a new valid url.

-- 
Brian Dolbec <dolsen>

[gentoo-dev] Re: git://anongit.gentoo.org is extremely slow

[Duncan]

Brian Dolbec posted on Wed, 22 Apr 2015 20:38:15 -0700 as excerpted:

> On Thu, 23 Apr 2015 15:20:15 +1200 Kent Fredric <kentfredric@gmail.com>
> wrote:
> 
>> On 23 April 2015 at 15:07, Duncan <1i5t5.duncan@cox.net> wrote:
>> 
>> > Perhaps the slashdot effect of everybody almost at once needing to
>> > delete and (re)add their overlays in layman, thus forcing a new
>> > clone,
>> >
>> >
>> Eh? That seems incredibly wrong for git.
>> 
>> git remote set-url master <newurl>
>> 
>> Would handle the git side.
>> 
>> You can even tweak git to rewrite all urls for you, like:
>> 
>> [url "git://anongit.gentoo.org"]
>>     insteadOf = git://git.overlays.gentoo.org
>> 
>> Admittedly it may have been hard for layman to be intelligent about
>> this, but requiring everyone to remove/add simply for a URI change
>> strikes me as rather odd here.
> 
> No, layman does try to update the url first, failing that it
> deletes/re-adds it.
> 
> But I think what happened for some, was, the urls had not yet been
> updated or updated correctly in repositories.xml.  That resulted in
> failures to update to a new valid url.

What I know is I was getting a sync-error (IIRC 1) for the overlays, and 
that deleting/re-adding the overlays via layman, fixed it.

If repositories.xml was not yet updated, then how would deleting/re-
adding have worked?

The overlays were the x11 and kde overlays, both git, both set in 
repos.conf to sync-type = laymansync, with the syncing done via script 
that calls emaint sync --repo (so syncs can be backgrounded and done in 
parallel).


Meanwhile, from the bug reports it looks like this switch did trigger 
reports for a number of latent bugs in layman, with patches already in 
the live-git version, and I believe lessons were learned about 
coordinating repo updates when a major host changes as well, so 
hopefully, if there's a next time it'll go much smoother than this one 
did. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]

On 23 April 2015 at 21:02, Duncan <1i5t5.duncan@cox.net> wrote:

> with patches already in
> the live-git version, and I believe lessons were learned about
> coordinating repo updates when a major host changes as well, so
> hopefully, if there's a next time it'll go much smoother than this one
> did. =:^)
>

Fixed in the live git version they said.


>>> Emerging (1 of 1) app-portage/layman-9999::gentoo
>>> Unpacking source...
Cloning into bare repository '/usr/portage/distfiles/egit-src/layman.git'...
fatal: Unable to look up git.overlays.gentoo.org (port 9418) (Name or
service not known)



-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 1488 bytes --]

Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow

[Mike Gilbert]

On Sat, Apr 25, 2015 at 1:32 AM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> On 23 April 2015 at 21:02, Duncan <1i5t5.duncan@cox.net> wrote:
>>
>> with patches already in
>> the live-git version, and I believe lessons were learned about
>> coordinating repo updates when a major host changes as well, so
>> hopefully, if there's a next time it'll go much smoother than this one
>> did. =:^)
>
>
> Fixed in the live git version they said.
>
>
>>>> Emerging (1 of 1) app-portage/layman-9999::gentoo
>>>> Unpacking source...
> Cloning into bare repository '/usr/portage/distfiles/egit-src/layman.git'...
> fatal: Unable to look up git.overlays.gentoo.org (port 9418) (Name or
> service not known)

Fixed in CVS. ^_^

Re: [gentoo-dev] git://anongit.gentoo.org is extremely slow

[Robin H. Johnson]

On Wed, Apr 22, 2015 at 05:11:13PM +0300, Nikos Chantziaras wrote:
> Adding overlays with layman is now extremely slow after the overlays 
> were moved to anongit.gentoo.org. Cloning anything beginning with 
> "git://anongit.gentoo.org/" is capped at 15KB/s, sometimes 10KB/s.
> 
> This is unworkably slow.
> 
> Is this intentional?
Where are you located?

I'm wondering if you're suffering connectivity impacts of the location
change.

Both boxes have enough hardware:
anongit (manakin): Xeon E5-2620, 32GB RAM, 512GB SSD (RAID1), US-East
git (oystercatcher): Xeon E5-1650, 64GB RAM, 240GB SSD (RAID1), Germany

We'll be bringing a EU location for anongit online as well soon, mostly
as the US-East doesn't have IPv6, and we can't put a tunnel into there.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

[gentoo-dev] Re: git://anongit.gentoo.org is extremely slow

[Nikos Chantziaras]

On 23/04/15 23:01, Robin H. Johnson wrote:
> On Wed, Apr 22, 2015 at 05:11:13PM +0300, Nikos Chantziaras wrote:
>> Adding overlays with layman is now extremely slow after the overlays
>> were moved to anongit.gentoo.org. Cloning anything beginning with
>> "git://anongit.gentoo.org/" is capped at 15KB/s, sometimes 10KB/s.
>>
>> This is unworkably slow.
>>
>> Is this intentional?
> Where are you located?
>
> I'm wondering if you're suffering connectivity impacts of the location
> change.
>
> Both boxes have enough hardware:
> anongit (manakin): Xeon E5-2620, 32GB RAM, 512GB SSD (RAID1), US-East
> git (oystercatcher): Xeon E5-1650, 64GB RAM, 240GB SSD (RAID1), Germany
>
> We'll be bringing a EU location for anongit online as well soon, mostly
> as the US-East doesn't have IPv6, and we can't put a tunnel into there.

The problem seems to have been fixed now.

It was not a location problem, because only git:// seemed to be affected:

   git clone git://...    <- 15KB/s
   git clone https://...  <- 2MB/s

Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow

[Vadim A. Misbakh-Soloviov]

[-- Attachment #1: Type: text/plain, Size: 395 bytes --]

> It was not a location problem, because only git:// seemed to be affected:
> 
>    git clone git://...    <- 15KB/s
>    git clone https://...  <- 2MB/s

Looks like shaping on ISP side (i.e. it doesn't hear about Network 
Neutrality).

// Although, of course, it can be result server overloading and only infra-
team can investigate and tell real reason ;)

-- 
Best regards,
mva

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 819 bytes --]