Re: [gentoo-project] RFC: Dropping rsync as a tree distribution method

[Matthew Thode <prometheanfire@gentoo.org>]

[-- Attachment #1: Type: text/plain, Size: 1956 bytes --]

On 18-12-16 06:34:07, Rich Freeman wrote:
> On Sat, Dec 15, 2018 at 11:15 PM Alec Warner <antarus@gentoo.org> wrote:
> >
> > [1] Rich talked about some downsides earlier at https://lwn.net/Articles/759539/; but while these are challenges (some fixable) they are not necessarily blockers.
> 
> The thread has already touched on a few of those comments.  Despite
> only six months elapsing since I wrote that email, #1 no longer
> applies, and it sounds like #4 may not be as much of a concern.  As
> you've already stated #3 can be easily addressed - setting up a git
> mirror is very easy.
> 
> I think #2 is more of a fundamental design difference that probably
> will never go away.  If your tree is a year old then git WILL take
> longer and transfer more data than rsync.  My guess is that it will
> also cost more IO server-side than rsync, but it probably will be
> cheaper in CPU.  However, I bet that 95% of our users sync weekly or
> daily and in that use case it is going to go a lot faster, and
> probably be less mirror load as well, and it will be a TON less IO
> load on the client side.  I'm not sure how much IO cost there is to
> git garbage collection - that might offset this in the common shallow
> clone scenario.
> 
> I'd suggest that those with concerns give it a shot using Zac's
> suggested settings and see how it goes.  Really all you have to do is
> delete your local repo and adjust your sync settings and resync.  I
> think the local disk use is going to be the biggest source of user
> objection and I'm interested in what people observe here.
> 

I wonder if we can add a little logic to help at least a little bit on
the yearly syncers.  If over a 6 months, remove old git sync'd dir and
replance with new shallow clone?  Not perfect, but workable maybe.

Do we need to tell users to set up a git gc cron job or does portage
handle that for us now?

-- 
Matthew Thode (prometheanfire)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]