From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 2324 invoked from network); 1 Dec 2004 12:06:14 +0000 Received: from smtp.gentoo.org (156.56.111.197) by lists.gentoo.org with AES256-SHA encrypted SMTP; 1 Dec 2004 12:06:14 +0000 Received: from lists.gentoo.org ([156.56.111.196] helo=parrot.gentoo.org) by smtp.gentoo.org with esmtp (Exim 4.41) id 1CZTFO-00004J-2h for arch-gentoo-portage-dev@lists.gentoo.org; Wed, 01 Dec 2004 12:06:14 +0000 Received: (qmail 13750 invoked by uid 89); 1 Dec 2004 12:06:12 +0000 Mailing-List: contact gentoo-portage-dev-help@gentoo.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail Reply-To: gentoo-portage-dev@lists.gentoo.org X-BeenThere: gentoo-portage-dev@gentoo.org Received: (qmail 24205 invoked from network); 1 Dec 2004 12:06:12 +0000 From: Brian Harring Reply-To: ferringb@gentoo.org To: gentoo-portage-dev@lists.gentoo.org In-Reply-To: <200412011225.27593.g.guidi@sns.it> References: <9ef20ef3041127151046107fb5@mail.gmail.com> <200412011037.06170.g.guidi@sns.it> <1101898798.32056.124.camel@localhost.localdomain> <200412011225.27593.g.guidi@sns.it> Content-Type: text/plain Message-Id: <1101902921.32056.129.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Wed, 01 Dec 2004 04:08:41 -0800 Content-Transfer-Encoding: 7bit Subject: Re: [gentoo-portage-dev] Current portage well designed, but badly used X-Archives-Salt: 22ed3761-55cf-4393-b20b-f47f00857ff1 X-Archives-Hash: 0d30d88894fb0e79f6b15435e57fd161 On Wed, 2004-12-01 at 03:25, Gregorio Guidi wrote: > On Wednesday 01 December 2004 11:59, Brian Harring wrote: > > On Wed, 2004-12-01 at 01:37, Gregorio Guidi wrote: > > > On Tue, 30 Nov 2004 06:22:07 -0800, Brian Harring > > > > 1) don't modify $portdir/metadata/cache. Modifying the rsync'd cache > > directly results in more to rsync for the next sync. Going that route > > would result in a lot of dial up users out for blood. > > 2) the user may not be using the same cache backend as the distributed > > cache- the rsync'd cache is portage_db_flat, while the user may be using > > a custom sqlite backend. I suspect this will be come more common place > > whenever cvs becomes stabled- the cache backend is being refactored > > currently. > > 3) cleanse stale entries from the cache, so you don't end up w/ a couple > > thousand extra files sitting in your local cache. Stable portage > > accomplishes this by wiping *all* local cache entries, and transferring > > the *entire* rsync'd cache over. Cvs is smarter, transfers only what > > has changed, and removes the stale entries (this is why it's faster). > > I was interested exactly in points 1 and 3 (and I'm glad to hear there's even > a sqlite backend). Nothing official sqlite thus far, I just know a couple of people who are did their own DIY backend for it. The cache refactoring should include a sqlite backend, although it hasn't been coded yet. > With respect to point 3, the solution you imlpemented is surely the right > thing to do. > With respect to point 1, maybe a good solution could be to just check cache > validity in $portdir/metadata when the cache is needed, and write a new cache > in /var/cache/edb if it is not valid. Err... elaborate Assuming I'm following, you're proposing doing the updates to local cache, and reading from the metacache? If so, that's twice the level of potential stats/reads required, which won't speed it up much :) ~brian -- gentoo-portage-dev@gentoo.org mailing list