Re: [gentoo-portage-dev] portage doc & cache

["Nicholas Jones" <carpaski@twobit.net>]

[-- Attachment #1: Type: text/plain, Size: 2857 bytes --]

> I'm wondering where I can find some doc about portage, and especially 
> the cache part. I use debian at work, and get so sad when doing an 
> emerge rsync at home (10 or 15 minutes to rsync, and almost 5 minutes to 
> calculate world dependancies) on a bi-athlon 1.8GHz.

This can be highly dependent on portage version. If you are using
2.0.51, there is no cache available on the servers as of yet. The
format has changed.

Using 2.0.51, with an EMPTY cache the performace should still
be greater than what you are stating. It takes me 2m59.628 on
my P4-2.2 Laptop (5400 RPM) to 'emerge -puD world'. If you're
seeing 5 minutes, there may be an issue on your system with IO.

After caching, it takes 7.867 seconds on my laptop.

2.0.50 should be fairly similar to the rates of 2.0.51 with
a populated cache. The generation times can vary. A hidden
feature, 'the modules file', might help as well.

carpaski@fenchurch carpaski $ cat /etc/portage/modules
portdbapi.auxdbmodule="portage_db_anydbm.database"
eclass_cache.dbmodule="portage_db_anydbm.database"

You'll have to sync after you make those changes as the cache
is essentially invalidated/not-present.

> maybe ebuilds should be "compiled" into seriaziled objects
> or something similar. Only an emerge rsync (or something
> called "emerge update") would fetch new ebuilds, and compile
> new ones (we don't care anymore of hand modified files after sync).

There are a lot of things you could take for granted with the
cache. There are problems inherent with assumptions though; they
tend to break things. Portage forces a complete update of the
cache to ensure that nothing strange happens. The cache entries
_CAN_ and _DO_ change randomly sometimes. Fixing a portage bug
here and there might cause a variable to change and without the
regeneration you wouldn't see that. We've had a lot of issues in
the past where data in the tree was corrupt and/or wrong. This
way expends a couple more minutes on the user's part and can free
up many more on our dev's part.

> I don't think it would be really difficult to implement a server that 
> would handle requests like :

You would have to implement it. The problem is available time and
interfacing with existing systems. If it's not as easy, efficient,
and solid it's not readily acceptable.

> Moreover, do I REALLY need all ebuilds content ???? Why can't portage 
> just look at some headers (with arch, desc, etc.) and left the install 
> part on other files. Rsync local tree would be lighter, and faster to 
> sync. When you need to emerge some program, emerge would download the 
> _real_ ebuild file, the digests, the distfiles, etc.

It wouldn't be any faster. Rsync uses stat values to do the transfer
determination. The values would be exactly the same for headers-only.

--NJ


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]