[gentoo-portage-dev] About boosting sync

[gentoo-portage-dev] About boosting sync

[Tambet]

[-- Attachment #1: Type: text/plain, Size: 469 bytes --]

Has anyone ever noticed that portage tree contains a lot of md5 hashes,
which are not at all important for using it? I think that it does not make
reliability or functionality smaller any bit if those would all stay in sync
servers - anyway, syncing would go much faster and this tree smaller. What
about removing all those md5 hashes and downloading them only when they're
needed?

Tambet - technique evolves to art, art evolves to magic, magic evolves to
just doing.

[-- Attachment #2: Type: text/html, Size: 496 bytes --]

Re: [gentoo-portage-dev] About boosting sync

[Ned Ludd]

On Tue, 2008-12-02 at 19:46 +0200, Tambet wrote:
> Has anyone ever noticed that portage tree contains a lot of md5
> hashes, which are not at all important for using it? I think that it
> does not make reliability or functionality smaller any bit if those
> would all stay in sync servers - anyway, syncing would go much faster
> and this tree smaller. What about removing all those md5 hashes and
> downloading them only when they're needed?

To build a deptree portage needs to source the ebuild in the depend
phase, so portage needs to know that a file is safe to source before it
loads it. Being that FEATURES='strict' is enabled per default in all
profiles. It's rather vital that things remain the way they are now.


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

Re: [gentoo-portage-dev] About boosting sync

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1802 bytes --]

On Tue, Dec 02, 2008 at 07:46:13PM +0200, Tambet wrote:
> Has anyone ever noticed that portage tree contains a lot of md5 hashes,
> which are not at all important for using it? I think that it does not make
> reliability or functionality smaller any bit if those would all stay in sync
> servers - anyway, syncing would go much faster and this tree smaller. What
> about removing all those md5 hashes and downloading them only when they're
> needed?
Umm, what are you on? There are no more MD5s in Manifest2. It should be
only RMD160, SHA1, SHA256. If you DO find a Manifest with an MD5, I'd
REALLY like to know about it.

As for the important of Manifests and the hashes, I'd like to offer the
following as suggested reading:
http://www.cs.arizona.edu/people/justin/packagemanagersecurity/
Specifically, see the papers page, and find the paper from CCS 2008 [1].
He DID solicit input from me on how Gentoo deals with the issue, and
gave it fair coverage in my opinion. It's CRITICALLY important that the
checksums go with the content, and that the checksums are later verified
themselves against a known up to date source.

If you're interested in the Gentoo side of it, specifically how it ties
into tree-signing, read my gleps:
http://www.gentoo.org/proj/en/glep/glep-0057.html
http://www.gentoo.org/proj/en/glep/glep-0058.html
http://www.gentoo.org/proj/en/glep/glep-0059.html
http://www.gentoo.org/proj/en/glep/glep-0060.html
http://www.gentoo.org/proj/en/glep/glep-0061.html

[1] Cappos, J. et al. "A Look In the Mirror: Attacks on Package
Managers". (2008). Published in the proceedings of ACM CCS 2008.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --]