From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([69.77.167.62] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1LYKQ6-0006V1-QA for garchives@archives.gentoo.org; Sat, 14 Feb 2009 13:19:00 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 71B75E0387; Sat, 14 Feb 2009 13:18:56 +0000 (UTC) Received: from rv-out-0708.google.com (rv-out-0708.google.com [209.85.198.240]) by pigeon.gentoo.org (Postfix) with ESMTP id 38DE7E0387 for ; Sat, 14 Feb 2009 13:18:56 +0000 (UTC) Received: by rv-out-0708.google.com with SMTP id f25so1097984rvb.46 for ; Sat, 14 Feb 2009 05:18:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:date:from:to:subject :message-id:references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=La8OVI/lGUfdLjQCz9Jbvele7H+FufuyF25siWd7WBQ=; b=sZn36+OeG3MoLKMLm7cPrWg2bwJAiimPmO+OTRbDY7+P4a5+D26WB56Cpk9oB3OeQi mXIG/GLe239dHe+9BfE4/dsie858VS4wf2aZpSSxRs1ZQvx9InBF9d5WLoDP2e/6bqCB qd9dQC7q4Wf2ZbATtMOw5ieNx0XJsLiv+EnOQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=DQbOfWiIxSieK9LznYZdxohVYd21FgmcjRcnKgSe0QXv/bnYxPmYQIbl+hKELdbh8j J6TXELuzB663ZuPxq7W9L7eVwWoz1jM+wSaZ+3qz21ChoV7N0EEUW+fo9RuhqIu0WQFb fTwyAztKh8H1+BDU4F6cm1SFf+KuAsgaYrcfE= Received: by 10.142.77.11 with SMTP id z11mr5897wfa.277.1234617535697; Sat, 14 Feb 2009 05:18:55 -0800 (PST) Received: from smtp.gmail.com (c-98-210-196-21.hsd1.ca.comcast.net [98.210.196.21]) by mx.google.com with ESMTPS id 24sm6061327wfc.57.2009.02.14.05.18.52 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 14 Feb 2009 05:18:54 -0800 (PST) Received: by smtp.gmail.com (sSMTP sendmail emulation); Sat, 14 Feb 2009 05:18:50 -0800 Date: Sat, 14 Feb 2009 05:18:50 -0800 From: Brian Harring To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] [RFC] DIGESTS metadata variable for cache validation Message-ID: <20090214131850.GA13200@hrair> References: <498758E6.5080609@gentoo.org> <1234045916.24784.1373.camel@localhost> <498E17E6.8060407@gentoo.org> <1234192940.18160.1011.camel@localhost> <49908A3D.4050403@gentoo.org> <20090210122046.GD4076@hrair> <4991E9D7.6080706@gentoo.org> <20090211090041.GA3680@hrair.corp.631h.metaweb.com> <4992A1F4.2040502@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline In-Reply-To: <4992A1F4.2040502@gentoo.org> User-Agent: Mutt/1.5.16 (2007-06-09) X-Archives-Salt: 98fd4e3a-6659-4187-aebe-57d4ad7bf137 X-Archives-Hash: 17041b113516483690c33b43a5fb8e81 --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 11, 2009 at 02:01:24AM -0800, Zac Medico wrote: > Brian Harring wrote: > > On Tue, Feb 10, 2009 at 12:55:51PM -0800, Zac Medico wrote: > >> Brian Harring wrote: > >>> Frankly, forget compatibility- the current format could stand to die.= =20 > >>> The repository format is an ever growing mess- leave it as is and=20 > >>> work on cutting over to something sane. > >> Changing the repository layout is a pretty radical thing to do. > >> You're welcome to start a new subject for that if you'd like but I'd > >> prefer to keep the scope of this thread focussed on the cache format > >> for the existing repository layout. >=20 > I don't intend to repeal the cache mtime requirement, at least > (especially) not on gentoo's rsync tree. However, I wouldn't say > that it's something that necessarily needs to be a requirement for > other repositories or overlays, moving forward (assuming that an > alternative validation framework is in place). So... you want a subset of repositories to have cache algo x, while=20 the rest have the old algo. And since the repo w/ algo x isn't=20 marked in some fashion, all managers will have to use new algo x for=20 compatibility reasons. Right... > > I reiterate, this belongs in a seperate repository format, along w/=20 > > the rest of the unversioned repository changes you've been pushing in= =20 > > (profile package.mask breaking all non portage PMs is a perfect=20 > > example). >=20 > The package.mask thing is a separate discussion. Let's do that in a > separate thread. Package.mask is relevant purely as a demonstration of why unversioned=20 changes to the repository formats *needs* to stop. Generally speaking=20 it's pretty shitty behaviour to embrace/extend a format when others=20 rely on it for interop. The annoying thing about this thread is that *no where* am I saying=20 you shouldn't be free to experiment. All I'm stating is that the end=20 result isn't a compatible repo- it *is* a new format (version even)=20 thus mark it in some way so that the rest of us can start properly=20 handling it rather then having to cut last minute releases since we're=20 PMS compliant but portage treats PMS as a subset of it's format rules. Pretty simple request, and not something that shouuld require argument=20 as far as I'm concerned. > > The daft thing about this is that w/ effectively atomic sync (if the=20 > > sync fails then mark the repo as screwed up till a sync completes),=20 > > the current cache format can *still* do validation- no clue if=20 > > paludis has it, but at least pkgcore and portage can handle this via=20 > > awareness of the eclass stacking. >=20 > I want to have a more fault-tolerant solution than that. I understand your reasoning, and frankly I used to view the rsync=20 issue in the same way- it's a naive view however since it implicitly=20 is assuming that the resultant repo is *usable*, iow that the actual=20 ebuild/eclass/profile data is valid, just that the updating bailed=20 during metadata transfer. There is zero gurantee as to where the=20 rsync bailed- meaning you can be missing patches, have trashed=20 manifests, etc. Well aware it's not friendly to require people to force a completed=20 sync before being able to use the repo, but it really is the only=20 *safe* option- as such the fault tolerant counterarg is a non=20 arguement. > > Note that proper PM implementations *still* have to set the cache=20 > > entries mtime for backwards compatibility w/ older PMs that don't=20 > > support this new unversioned change thus muddying the implementation=20 > > even further. >=20 > As said above, I wasn't intending that, at least (especially) not > for gentoo's rsync tree. I guess you got that idea from the mention > of bug 139134, but you don't need to worry about it. Implicitly it's required; if pkgcore is to generate cache entries for=20 repo x, it has to do exactly as I said so that any any pre=20 cache-modified-managers are still able to use the cache. That's=20 assuming the $PM cares about compatibility... ~harring --W/nzBZO5zC0uMSeA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkmWxLoACgkQsiLx3HvNzgenRACeOEqfMMN8R6581E0HMU2k3SK4 35MAnj6RGpCURwS4n8nwGCugaA0kzDi+ =4dak -----END PGP SIGNATURE----- --W/nzBZO5zC0uMSeA--