From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id D96631388C0 for ; Sat, 27 Feb 2016 10:31:07 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 296E721C064; Sat, 27 Feb 2016 10:30:57 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 4FD0421C001 for ; Sat, 27 Feb 2016 10:30:56 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1aZc9F-0004W0-Kj for gentoo-dev@lists.gentoo.org; Sat, 27 Feb 2016 11:30:53 +0100 Received: from pc123.math.cas.cz ([147.231.88.123]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 27 Feb 2016 11:30:53 +0100 Received: from martin by pc123.math.cas.cz with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 27 Feb 2016 11:30:53 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-dev@lists.gentoo.org From: Martin Vaeth Subject: [gentoo-dev] Re: Bug #565566: Why is it still not fixed? Date: Sat, 27 Feb 2016 10:30:45 +0000 (UTC) Message-ID: References: <56CC937C.3030805@gentoo.org> <56CCD4DC.3040509@gentoo.org> <56CCFE65.5050201@gentoo.org> <20160224211613.51a613ff@gentp.lnet> X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: pc123.math.cas.cz User-Agent: slrn/1.0.1 (Linux) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Archives-Salt: a23007b0-28ff-40d6-8d47-88cac05cf4f4 X-Archives-Hash: 04db328edf2f17fb0ce0c11ed9c9e7fd Rich Freeman wrote: > > Clearly it doesn't increase by a factor of 1 every year The yearly increase of the factor is rather precisely 1: According to current data, it is .95, see below. With xz compression for squashfs, it is even 1.4! (Note: increase _of_ the factor, not _by_ the factor, of course; we are speaking about a linear increase, not an exponential one.) More precisely: If in both cases you extremeley optimize for space (details see below) then a change from rsync to git (non-shallow) costs you a) now: the factor 2.6 of needed disk space b) in future for every year this factor is increased by the summand 1.4. For example, in 2.5 years you will need roughly 2.6 + (1.4 * 2.5) = 6.1 times the disk space than for rsync. After 2.5 more years, the factor will be more than 10. For a) I assumed that in both cases the current repository is kept compressed with squashfs (xz). This first factor will be much larger, of course, if you omit squashfs when you switch to git. (You must take measurements to keep the checked-out repository separate: you cannot use standard emerge --sync to get this optimization.) For both numbers, I even optimized the .git compression by executing repeatedly git prune; git repack -a -d; git gc --agressive which for the historical repository took several hours; thus, unless you use a cron-job, this is not realistic. Without this optimization, both numbers would be even larger. Here are the plain data I used for the calculation: 1. RSYNC = 84,062,208 (rsync gentoo repository, compressed with squashfs (-comp xz).) 2. GIT = 136,322,616 (Current .git data, without checked-out tree; compression optimized by the time-costly commands above.) 3. FULL = 1,923,685,435 (.git data as in 2, but with history added) 4. YEARS = 15 (length of the historical data: first checkin was June 2000; change to git was IIRC somewhere in middle 2015). So the number from a) is size with git $GIT + $RSYNC --------------- = ------------- ~ 2.6 size with rysnc $RSYNC The number from b) is size of history increase per year ($FULL - $GIT) / $YEARS --------------------------------- = ------------------------ ~ 1.4 size with rsync $RSYNC In the previus postings, I was assuming the much faster squashfs compression -comp lz4 -Xhc instead of -comp xz. In this case, the number from 1 changes to RSYNC = 125784064 which leads to the factor .95 ~ 1 for b) which I mentioned in the beginning.