From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 16433 invoked from network); 7 Feb 2004 17:04:13 +0000 Received: from smtp.gentoo.org (128.193.0.39) by eagle.gentoo.oregonstate.edu with DES-CBC3-SHA encrypted SMTP; 7 Feb 2004 17:04:13 +0000 Received: from lists.gentoo.org ([128.193.0.34] helo=eagle.gentoo.org) by smtp.gentoo.org with esmtp (Exim 4.24) id 1ApVsI-0002yO-Kt for arch-gentoo-portage-dev@lists.gentoo.org; Sat, 07 Feb 2004 17:04:10 +0000 Received: (qmail 23230 invoked by uid 50004); 7 Feb 2004 17:04:09 +0000 Mailing-List: contact gentoo-portage-dev-help@gentoo.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail Reply-To: gentoo-portage-dev@lists.gentoo.org X-BeenThere: gentoo-portage-dev@gentoo.org Received: (qmail 1751 invoked from network); 7 Feb 2004 17:04:09 +0000 Date: Sat, 7 Feb 2004 18:04:18 +0100 To: gentoo-portage-dev@lists.gentoo.org Message-ID: <20040207170418.GA558@hx.se> References: <200402071318.47224.thomas@horsten.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200402071318.47224.thomas@horsten.com> User-Agent: Mutt/1.3.28i From: Simon Mika Subject: Re: [gentoo-portage-dev] emerge feature request: Downloads managed by lock file system X-Archives-Salt: 654ed166-7922-4104-94b0-8ddd61f64f1a X-Archives-Hash: a01c3557dbb36a5876a14c471f9decca Although I have a computer that is a bit slower I have the similiar problem. Downloading does not consume much processorpower and athor machine resources. But compiling does need a lot. Wy don't use always use 2 parallell processes, one that downloads, and one that compiles. The compiling process would still need to be synced with the downloading, but that should not be too hard to implement with some semaphore or similiar solution. /Simon Mika On Sat, Feb 07, 2004 at 01:18:47PM +0000, Thomas Horsten wrote: > Hi, > > I have a request for an emerge feature that I think would be useful for a lot > of people. > > I have quite a fast machine with RAID and 3GHz CPU, and only a 512kbit/s > Internet connection. When I do an "emerge -u world" most of the time actually > goes by waiting for packages to download. For most ebuilds, It takes roughly > the same time to download as to compile. > > So to speed up my builds, I usually start an "emerge -f -u world". Once the > first couple of packages have downloaded I then start an "emerge -u world" in > another shell. > > This works great as long as the fetcher process is ahead of the builder > process. But if some packages are built too quickly, the builder process > catches up. It then starts downloading the same file as the downloader > process, resuming from wherever the downloader had gotten to, so now two > downloaders are writing to the same file (waste of bandwidth and a possible > source of corrupted archives). > > What I suggest is that emerge will put a lockfile in place when it's > downloading a file, eg. "name-of-package.tar.bz2.lock", or by using fcntl > locks. Then if the "builder" catches up, and finds the output file is already > open, it waits on the lock for the download to complete and then continues > unpacking after that. > > I already love gentoo and the portage system, it's by far the best available, > but adding this feature would make it even better IMHO. > > Regards, > > Thomas > > > -- > gentoo-portage-dev@gentoo.org mailing list > -- gentoo-portage-dev@gentoo.org mailing list