[gentoo-portage-dev] emerge feature request: Downloads managed by lock file system

[Thomas Horsten <thomas@horsten.com>]

Hi,

I have a request for an emerge feature that I think would be useful for a lot 
of people.

I have quite a fast machine with RAID and 3GHz CPU, and only a 512kbit/s 
Internet connection. When I do an "emerge -u world" most of the time actually 
goes by waiting for packages to download. For most ebuilds, It takes roughly 
the same time to download as to compile.

So to speed up my builds, I usually start an "emerge -f -u world". Once the 
first couple of packages have downloaded I then start an "emerge -u world" in 
another shell.

This works great as long as the fetcher process is ahead of the builder 
process. But if some packages are built too quickly, the builder process 
catches up. It then starts downloading the same file as the downloader 
process, resuming from wherever the downloader had gotten to, so now two 
downloaders are writing to the same file (waste of bandwidth and a possible 
source of corrupted archives).

What I suggest is that emerge will put a lockfile in place when it's 
downloading a file, eg. "name-of-package.tar.bz2.lock", or by using fcntl 
locks. Then if the "builder" catches up, and finds the output file is already 
open, it waits on the lock for the download to complete and then continues 
unpacking after that.

I already love gentoo and the portage system, it's by far the best available, 
but adding this feature would make it even better IMHO.

Regards,

Thomas


--
gentoo-portage-dev@gentoo.org mailing list