From: "Robin H. Johnson" <robbat2@gentoo.org>
To: gentoo-scm@lists.gentoo.org
Subject: [gentoo-scm] Progress summary, 2009/06/01
Date: Mon, 1 Jun 2009 16:45:49 -0700 [thread overview]
Message-ID: <20090601234549.GA3042@orbis-terrarum.net> (raw)
[-- Attachment #1: Type: text/plain, Size: 3874 bytes --]
Update on this TODO list, and extension for new items.
I've deliberately broken the threading so that more people read it. This
was a response to the mail with the subject of "Converting a recent CVS
copy - Item 2: statistics"
Executive summary:
- We've gone from 18.5 hours to 9 hours, all in a single portion of the
conversion, thanks to help from upstream. There's lots more room for
improvement.
- C (for git) and Python (for cvs2svn) coders very welcome to challenge
the problems.
- Testers wanted
- Actually forming this up to a project with a team is probably due
soon. I've had interest/direct offers of help from: WilliamH, Calchan,
Betelgeuse over the last week.
If you want something, pick it from this mail, and try to flesh it out
on the list with me while you work on it.
New TODO items:
- Finish new hooks for git:
upload-pack.c - get_common_commits:
right before the 'return 0', check for and launch a new hook, passing
the have/want headers via stdin, and use the return code (AND the
stderr) to see if we should halt.
- Review commit signing
- pclouds (a former Gentoo dev) contributed this prototype:
http://thread.gmane.org/gmane.comp.version-control.git/115562/focus=118788
- I'm not entirely convinced the above is right, as the commit message
seems to end up unsigned.
- Wait for the commit-notes patches onto upstream Git?
- Test git-cvsserver usage.
- Both remote and local modes.
- mips and narrow checkouts may want this.
On Tue, Apr 14, 2009 at 01:33:24AM -0700, Robin H. Johnson wrote:
> TODO:
> - Could somebody with Python-foo please look hard at cvs2svn with an eye to
> making it multi-threaded?
> - Focus on pass1 and pass9.
> - pass8 maybe as well, but I think it will be harder by design.
- pass1 optimization was completed 20 minutes ago by mhagger (one of the
cvs2svn upstream lead developers). From an old time of 36204 seconds,
it's now only 1598 seconds (and we only used 2 CPU cores so far, we
have 6 more for later). 22x speedup :-).
- mhagger from upstream needed hardware to test on, so I hooked him up
with access to the experimental conversion box.
- TODO: pass9/pass8 remain.
- Need to validate output of new mode against the previous mode.
> - We need incremental conversion stuff badly.
Incremental may become practically unneeded if we can get the conversion
time under 2 hours.
> - I had to use the RCSRevisionReader, as InternalRevisionReader seemed to be
> broken. Would make pass1 faster as well.
This is now completely unneeded. The pass1 solution integrated the
rcsparse code into the new generate_blobs.py.
> - Should probably ignore the '.frozen' files.
No support to exclude files presently.
TODO: Implement inside def _generate_cvs_files
> - Review RCS state of ALL ,v files. There are a few non-dead files in Attic.
Thanks to William Hubbs (williamh) for his review. I've got a few more
items to process from this review.
> - Maybe trim out the Manifest/digest contents during the conversion, leaving
> only DIST lines?
> Con: _WOULD_ break old GPG signatures.
> Pro: probably help size a lot.
Additional con:
- I think it will massively slows down the conversion right now.
- Nothing more than a bad prototype I did. I'd like somebody else to
attack the problem without having seen my prototype, rather than be
infected with my bad ideas in it.
Upstream misc:
- Subtree checkouts
- Is progressing upstream. Now known as checkout modes: narrow,
sparse, shallow
= narrow: some directory that is not the root.
= sparse: a subset of files in a directory.
= shallow: subset of recent history.
--
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85
[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]
next reply other threads:[~2009-06-01 23:46 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-06-01 23:45 Robin H. Johnson [this message]
2009-06-05 12:59 ` [gentoo-scm] gpg signing of commits, was: Progress summary, 2009/06/01 Robert Buchholz
2009-06-05 18:54 ` Robin H. Johnson
2009-06-09 1:50 ` Robert Buchholz
2009-06-09 2:46 ` Robin H. Johnson
2009-06-07 22:18 ` [gentoo-scm] " Donnie Berkholz
2009-06-07 22:27 ` Nirbheek Chauhan
2009-06-09 19:42 ` [gentoo-scm] New hooks for git, was: " Arun Raghavan
2009-06-09 20:16 ` Robin H. Johnson
2009-08-27 22:59 ` [gentoo-scm] Progress summary, 2009/08/27 Robin H. Johnson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090601234549.GA3042@orbis-terrarum.net \
--to=robbat2@gentoo.org \
--cc=gentoo-scm@lists.gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox