Re: [gentoo-project] RFC: Dropping rsync as a tree distribution method

[Zac Medico <zmedico@gentoo.org>]

[-- Attachment #1.1: Type: text/plain, Size: 2653 bytes --]

On 12/15/18 9:17 PM, Alec Warner wrote:
> 
> 
> On Sun, Dec 16, 2018 at 12:13 AM Georgy Yakovlev <gyakovlev@gentoo.org
> <mailto:gyakovlev@gentoo.org>> wrote:
> 
>     On Saturday, December 15, 2018 8:40:38 PM PST Matt Turner wrote:
>     > On Sat, Dec 15, 2018 at 11:16 PM Alec Warner <antarus@gentoo.org
>     <mailto:antarus@gentoo.org>> wrote:
>     > > - Disk usage for git vs rsync
>     >
>     > This is why I have not switched. With git you pull down increasing
>     > amounts of history, whereas with rsync the data fits easily in a <1GB
>     > partition.
> 
>     Recent portage can use sync-depth = 1
>     repo dir no longer grows as it used to and it's works fine unlike
>     initial
>     implementation that was giving trouble
> 
>     https://bugs.gentoo.org/552814
> 
>     du -hs /var/db/repos/gentoo
>     350M    /var/db/repos/gentoo
> 
>     example /etc/portage/repos.conf/gentoo.conf :
>     [DEFAULT]
>     main-repo = gentoo
> 
>     [gentoo]
>     auto-sync = yes
>     location = /var/db/repos/gentoo
>     sync-type = git
>     sync-uri = https://github.com/gentoo-mirror/gentoo.git
>     sync-depth = 1
>     sync-git-clone-extra-opts = -b master
>     sync-git-verify-commit-signature = true
> 
> 
>     sync is almost instantaneous compared to rsync, but some folks not
>     going to
>     like github as a mirror in this case.
> 
> 
> I don't plan on using github for the mirror, so I'm not overly worried
> about that portion.
> 
> +Zac Medico <mailto:zmedico@gentoo.org> 
> 
> My recollection was that git doesn't ship with ebuild metadata by
> default, so even if we make the first sync fast (by using depth=1 in the
> clone) do we have a good story for ebuild metadata? Is portage just
> faster than in the past for ebuilds with missing metadata? Does emerge
> --sync handle metadata regen for syncs with git origins?
> 
> -A

The metadata has to be included in the git repostory, and we've
currently got "master" and "stable" branches which include everything
that the rsync tree has:

https://gitweb.gentoo.org/repo/sync/gentoo.git/log/?h=master
https://gitweb.gentoo.org/repo/sync/gentoo.git/log/?h=stable

Both branches are also mirrored on github:

https://github.com/gentoo-mirror/gentoo/commits/master
https://github.com/gentoo-mirror/gentoo/commits/stable

It would be interesting to see some garbage collection stats for
sync-deph = 1, people using it should post the output of this command:

git count-objects -v

>  
> 
> 
> 
>     -- 
>     Georgy Yakovlev
>     Gentoo Linux Developer
> 


-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]