Re: [gentoo-dev] Re: proxy maintainership and gentoo-x86 scm

["Robin H. Johnson" <robbat2@gentoo.org>]

[-- Attachment #1: Type: text/plain, Size: 2132 bytes --]

On Thu, Jan 14, 2010 at 07:07:01PM -0500, Paul Arthur wrote:
> On 2010-01-14, Nirbheek Chauhan <nirbheek@gentoo.org> wrote:
> > On Wed, Jan 13, 2010 at 9:24 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> >> i think our current work flows also significantly impede the smooth running of
> >> this.  if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
> >> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
> >> their behalf.
> >
> > In theory, yes. In practice, git is too slow to handle 30,000 files.
> > Even simple operations like git add become painful even if you put the
> > whole of portage on tmpfs since git does a stat() on every single file
> > in the repository with every operation.
> >
> > Simple test: do a git init followed by git add && git commit -m
> > "Initial commit" in your portage dir (.gitignore packages/ and
> > distfiles/)
> >
> > Once this is done, you can test how it'll feel like to use a DSCM on
> > portage (without history). Unless you have a really fast SSD and
> > processor, you'll want to go back to the good old days of CVS with its
> > network-bound latencies on just 5-6 files in the current dir.
> 
> Ouch.  I wanted to test this in a fairly bad scenario, so I gave it a
> try on my old, low-spec fileserver.
You didn't repack or at least run git-gc between the huge add and your
everyday operations. Do that, and then measure the ops with both cold
and hot cache. 

The initial packing and adding are very intensive, even on fast
machines, but that's because they are dealing with a lot of small pieces
of data. Packing has benefited immensely from being fully multi-threaded
in Git.

I'd love somebody to do the SoC stats again:
http://www.gentoo.org/proj/en/infrastructure/cvs-migration.xml?style=printable

Using the git repo conversion I did:
http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]