Re: [gentoo-portage-dev] emerge feature request: Downloads managed by lock file system

[Simon Mika <smika@hx.se>]

Although I have a computer that is a bit slower I have the similiar problem.
Downloading does not consume much processorpower and athor machine
resources. But compiling does need a lot. Wy don't use always use 2
parallell processes, one that downloads, and one that compiles. The
compiling process would still need to be synced with the downloading, but
that should not be too hard to implement with some semaphore or similiar
solution.

/Simon Mika


On Sat, Feb 07, 2004 at 01:18:47PM +0000, Thomas Horsten wrote:
> Hi,
> 
> I have a request for an emerge feature that I think would be useful for a lot 
> of people.
> 
> I have quite a fast machine with RAID and 3GHz CPU, and only a 512kbit/s 
> Internet connection. When I do an "emerge -u world" most of the time actually 
> goes by waiting for packages to download. For most ebuilds, It takes roughly 
> the same time to download as to compile.
> 
> So to speed up my builds, I usually start an "emerge -f -u world". Once the 
> first couple of packages have downloaded I then start an "emerge -u world" in 
> another shell.
> 
> This works great as long as the fetcher process is ahead of the builder 
> process. But if some packages are built too quickly, the builder process 
> catches up. It then starts downloading the same file as the downloader 
> process, resuming from wherever the downloader had gotten to, so now two 
> downloaders are writing to the same file (waste of bandwidth and a possible 
> source of corrupted archives).
> 
> What I suggest is that emerge will put a lockfile in place when it's 
> downloading a file, eg. "name-of-package.tar.bz2.lock", or by using fcntl 
> locks. Then if the "builder" catches up, and finds the output file is already 
> open, it waits on the lock for the download to complete and then continues 
> unpacking after that.
> 
> I already love gentoo and the portage system, it's by far the best available, 
> but adding this feature would make it even better IMHO.
> 
> Regards,
> 
> Thomas
> 
> 
> --
> gentoo-portage-dev@gentoo.org mailing list
> 

--
gentoo-portage-dev@gentoo.org mailing list