* [gentoo-dev] git://anongit.gentoo.org is extremely slow @ 2015-04-22 14:11 Nikos Chantziaras 2015-04-23 3:07 ` [gentoo-dev] " Duncan 2015-04-23 20:01 ` [gentoo-dev] " Robin H. Johnson 0 siblings, 2 replies; 10+ messages in thread From: Nikos Chantziaras @ 2015-04-22 14:11 UTC (permalink / raw To: gentoo-dev Adding overlays with layman is now extremely slow after the overlays were moved to anongit.gentoo.org. Cloning anything beginning with "git://anongit.gentoo.org/" is capped at 15KB/s, sometimes 10KB/s. This is unworkably slow. Is this intentional? ^ permalink raw reply [flat|nested] 10+ messages in thread
* [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow 2015-04-22 14:11 [gentoo-dev] git://anongit.gentoo.org is extremely slow Nikos Chantziaras @ 2015-04-23 3:07 ` Duncan 2015-04-23 3:20 ` Kent Fredric 2015-04-23 20:01 ` [gentoo-dev] " Robin H. Johnson 1 sibling, 1 reply; 10+ messages in thread From: Duncan @ 2015-04-23 3:07 UTC (permalink / raw To: gentoo-dev Nikos Chantziaras posted on Wed, 22 Apr 2015 17:11:13 +0300 as excerpted: > Adding overlays with layman is now extremely slow after the overlays > were moved to anongit.gentoo.org. Cloning anything beginning with > "git://anongit.gentoo.org/" is capped at 15KB/s, sometimes 10KB/s. > > This is unworkably slow. > > Is this intentional? Perhaps the slashdot effect of everybody almost at once needing to delete and (re)add their overlays in layman, thus forcing a new clone, in ordered to kill the sync errors due to the previously configured host disappearing? I had that happen a couple days ago and once I figured out that the old host shutdowns was why I was getting the errors, the delete/re-add process went reasonably smoothly, but that might have been before the big rush. Or maybe it's simply suffering an attack DDoS? Infra? FWIW, once cloned, updates seem to be reasonably normal/fast, at least for the two overlays (kde and x11) I have subscribed. Maybe they're limiting cloning speed due to the above, figuring people will get thru it eventually, to keep normal updates reasonable? -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow 2015-04-23 3:07 ` [gentoo-dev] " Duncan @ 2015-04-23 3:20 ` Kent Fredric 2015-04-23 3:38 ` Brian Dolbec 0 siblings, 1 reply; 10+ messages in thread From: Kent Fredric @ 2015-04-23 3:20 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 679 bytes --] On 23 April 2015 at 15:07, Duncan <1i5t5.duncan@cox.net> wrote: > Perhaps the slashdot effect of everybody almost at once needing to delete > and (re)add their overlays in layman, thus forcing a new clone, > Eh? That seems incredibly wrong for git. git remote set-url master <newurl> Would handle the git side. You can even tweak git to rewrite all urls for you, like: [url "git://anongit.gentoo.org"] insteadOf = git://git.overlays.gentoo.org Admittedly it may have been hard for layman to be intelligent about this, but requiring everyone to remove/add simply for a URI change strikes me as rather odd here. -- Kent *KENTNL* - https://metacpan.org/author/KENTNL [-- Attachment #2: Type: text/html, Size: 1716 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow 2015-04-23 3:20 ` Kent Fredric @ 2015-04-23 3:38 ` Brian Dolbec 2015-04-23 9:02 ` Duncan 0 siblings, 1 reply; 10+ messages in thread From: Brian Dolbec @ 2015-04-23 3:38 UTC (permalink / raw To: gentoo-dev On Thu, 23 Apr 2015 15:20:15 +1200 Kent Fredric <kentfredric@gmail.com> wrote: > On 23 April 2015 at 15:07, Duncan <1i5t5.duncan@cox.net> wrote: > > > Perhaps the slashdot effect of everybody almost at once needing to > > delete and (re)add their overlays in layman, thus forcing a new > > clone, > > > > Eh? That seems incredibly wrong for git. > > git remote set-url master <newurl> > > Would handle the git side. > > You can even tweak git to rewrite all urls for you, like: > > [url "git://anongit.gentoo.org"] > insteadOf = git://git.overlays.gentoo.org > > Admittedly it may have been hard for layman to be intelligent about > this, but requiring everyone to remove/add simply for a URI change > strikes me as rather odd here. > > No, layman does try to update the url first, failing that it deletes/re-adds it. But I think what happened for some, was, the urls had not yet been updated or updated correctly in repositories.xml. That resulted in failures to update to a new valid url. -- Brian Dolbec <dolsen> ^ permalink raw reply [flat|nested] 10+ messages in thread
* [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow 2015-04-23 3:38 ` Brian Dolbec @ 2015-04-23 9:02 ` Duncan 2015-04-25 5:32 ` Kent Fredric 0 siblings, 1 reply; 10+ messages in thread From: Duncan @ 2015-04-23 9:02 UTC (permalink / raw To: gentoo-dev Brian Dolbec posted on Wed, 22 Apr 2015 20:38:15 -0700 as excerpted: > On Thu, 23 Apr 2015 15:20:15 +1200 Kent Fredric <kentfredric@gmail.com> > wrote: > >> On 23 April 2015 at 15:07, Duncan <1i5t5.duncan@cox.net> wrote: >> >> > Perhaps the slashdot effect of everybody almost at once needing to >> > delete and (re)add their overlays in layman, thus forcing a new >> > clone, >> > >> > >> Eh? That seems incredibly wrong for git. >> >> git remote set-url master <newurl> >> >> Would handle the git side. >> >> You can even tweak git to rewrite all urls for you, like: >> >> [url "git://anongit.gentoo.org"] >> insteadOf = git://git.overlays.gentoo.org >> >> Admittedly it may have been hard for layman to be intelligent about >> this, but requiring everyone to remove/add simply for a URI change >> strikes me as rather odd here. > > No, layman does try to update the url first, failing that it > deletes/re-adds it. > > But I think what happened for some, was, the urls had not yet been > updated or updated correctly in repositories.xml. That resulted in > failures to update to a new valid url. What I know is I was getting a sync-error (IIRC 1) for the overlays, and that deleting/re-adding the overlays via layman, fixed it. If repositories.xml was not yet updated, then how would deleting/re- adding have worked? The overlays were the x11 and kde overlays, both git, both set in repos.conf to sync-type = laymansync, with the syncing done via script that calls emaint sync --repo (so syncs can be backgrounded and done in parallel). Meanwhile, from the bug reports it looks like this switch did trigger reports for a number of latent bugs in layman, with patches already in the live-git version, and I believe lessons were learned about coordinating repo updates when a major host changes as well, so hopefully, if there's a next time it'll go much smoother than this one did. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow 2015-04-23 9:02 ` Duncan @ 2015-04-25 5:32 ` Kent Fredric 2015-04-25 13:58 ` Mike Gilbert 0 siblings, 1 reply; 10+ messages in thread From: Kent Fredric @ 2015-04-25 5:32 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 656 bytes --] On 23 April 2015 at 21:02, Duncan <1i5t5.duncan@cox.net> wrote: > with patches already in > the live-git version, and I believe lessons were learned about > coordinating repo updates when a major host changes as well, so > hopefully, if there's a next time it'll go much smoother than this one > did. =:^) > Fixed in the live git version they said. >>> Emerging (1 of 1) app-portage/layman-9999::gentoo >>> Unpacking source... Cloning into bare repository '/usr/portage/distfiles/egit-src/layman.git'... fatal: Unable to look up git.overlays.gentoo.org (port 9418) (Name or service not known) -- Kent *KENTNL* - https://metacpan.org/author/KENTNL [-- Attachment #2: Type: text/html, Size: 1488 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow 2015-04-25 5:32 ` Kent Fredric @ 2015-04-25 13:58 ` Mike Gilbert 0 siblings, 0 replies; 10+ messages in thread From: Mike Gilbert @ 2015-04-25 13:58 UTC (permalink / raw To: Gentoo Dev On Sat, Apr 25, 2015 at 1:32 AM, Kent Fredric <kentfredric@gmail.com> wrote: > > On 23 April 2015 at 21:02, Duncan <1i5t5.duncan@cox.net> wrote: >> >> with patches already in >> the live-git version, and I believe lessons were learned about >> coordinating repo updates when a major host changes as well, so >> hopefully, if there's a next time it'll go much smoother than this one >> did. =:^) > > > Fixed in the live git version they said. > > >>>> Emerging (1 of 1) app-portage/layman-9999::gentoo >>>> Unpacking source... > Cloning into bare repository '/usr/portage/distfiles/egit-src/layman.git'... > fatal: Unable to look up git.overlays.gentoo.org (port 9418) (Name or > service not known) Fixed in CVS. ^_^ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-dev] git://anongit.gentoo.org is extremely slow 2015-04-22 14:11 [gentoo-dev] git://anongit.gentoo.org is extremely slow Nikos Chantziaras 2015-04-23 3:07 ` [gentoo-dev] " Duncan @ 2015-04-23 20:01 ` Robin H. Johnson 2015-04-24 14:38 ` [gentoo-dev] " Nikos Chantziaras 1 sibling, 1 reply; 10+ messages in thread From: Robin H. Johnson @ 2015-04-23 20:01 UTC (permalink / raw To: gentoo-dev On Wed, Apr 22, 2015 at 05:11:13PM +0300, Nikos Chantziaras wrote: > Adding overlays with layman is now extremely slow after the overlays > were moved to anongit.gentoo.org. Cloning anything beginning with > "git://anongit.gentoo.org/" is capped at 15KB/s, sometimes 10KB/s. > > This is unworkably slow. > > Is this intentional? Where are you located? I'm wondering if you're suffering connectivity impacts of the location change. Both boxes have enough hardware: anongit (manakin): Xeon E5-2620, 32GB RAM, 512GB SSD (RAID1), US-East git (oystercatcher): Xeon E5-1650, 64GB RAM, 240GB SSD (RAID1), Germany We'll be bringing a EU location for anongit online as well soon, mostly as the US-East doesn't have IPv6, and we can't put a tunnel into there. -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead E-Mail : robbat2@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 ^ permalink raw reply [flat|nested] 10+ messages in thread
* [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow 2015-04-23 20:01 ` [gentoo-dev] " Robin H. Johnson @ 2015-04-24 14:38 ` Nikos Chantziaras 2015-04-24 15:04 ` Vadim A. Misbakh-Soloviov 0 siblings, 1 reply; 10+ messages in thread From: Nikos Chantziaras @ 2015-04-24 14:38 UTC (permalink / raw To: gentoo-dev On 23/04/15 23:01, Robin H. Johnson wrote: > On Wed, Apr 22, 2015 at 05:11:13PM +0300, Nikos Chantziaras wrote: >> Adding overlays with layman is now extremely slow after the overlays >> were moved to anongit.gentoo.org. Cloning anything beginning with >> "git://anongit.gentoo.org/" is capped at 15KB/s, sometimes 10KB/s. >> >> This is unworkably slow. >> >> Is this intentional? > Where are you located? > > I'm wondering if you're suffering connectivity impacts of the location > change. > > Both boxes have enough hardware: > anongit (manakin): Xeon E5-2620, 32GB RAM, 512GB SSD (RAID1), US-East > git (oystercatcher): Xeon E5-1650, 64GB RAM, 240GB SSD (RAID1), Germany > > We'll be bringing a EU location for anongit online as well soon, mostly > as the US-East doesn't have IPv6, and we can't put a tunnel into there. The problem seems to have been fixed now. It was not a location problem, because only git:// seemed to be affected: git clone git://... <- 15KB/s git clone https://... <- 2MB/s ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [gentoo-dev] Re: git://anongit.gentoo.org is extremely slow 2015-04-24 14:38 ` [gentoo-dev] " Nikos Chantziaras @ 2015-04-24 15:04 ` Vadim A. Misbakh-Soloviov 0 siblings, 0 replies; 10+ messages in thread From: Vadim A. Misbakh-Soloviov @ 2015-04-24 15:04 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 395 bytes --] > It was not a location problem, because only git:// seemed to be affected: > > git clone git://... <- 15KB/s > git clone https://... <- 2MB/s Looks like shaping on ISP side (i.e. it doesn't hear about Network Neutrality). // Although, of course, it can be result server overloading and only infra- team can investigate and tell real reason ;) -- Best regards, mva [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2015-04-25 13:58 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-04-22 14:11 [gentoo-dev] git://anongit.gentoo.org is extremely slow Nikos Chantziaras 2015-04-23 3:07 ` [gentoo-dev] " Duncan 2015-04-23 3:20 ` Kent Fredric 2015-04-23 3:38 ` Brian Dolbec 2015-04-23 9:02 ` Duncan 2015-04-25 5:32 ` Kent Fredric 2015-04-25 13:58 ` Mike Gilbert 2015-04-23 20:01 ` [gentoo-dev] " Robin H. Johnson 2015-04-24 14:38 ` [gentoo-dev] " Nikos Chantziaras 2015-04-24 15:04 ` Vadim A. Misbakh-Soloviov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox