From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1NcDJ4-0004Ck-BH for garchives@archives.gentoo.org; Tue, 02 Feb 2010 07:36:18 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 5EC94E0B7E; Tue, 2 Feb 2010 07:36:14 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id F22DFE087F for ; Tue, 2 Feb 2010 07:35:49 +0000 (UTC) Received: from mail.isohunt.com (b01.ext.isohunt.com [208.71.112.51]) by smtp.gentoo.org (Postfix) with ESMTP id 7291E67B2F for ; Tue, 2 Feb 2010 07:35:49 +0000 (UTC) Received: (qmail 23511 invoked from network); 2 Feb 2010 07:35:42 -0000 Received: from tsi-static.orbis-terrarum.net (HELO grubbs.orbis-terrarum.net) (76.10.188.108) by mail.isohunt.com (qpsmtpd/0.33-dev on beta01) with (CAMELLIA256-SHA encrypted) ESMTPS; Tue, 02 Feb 2010 07:35:42 +0000 Received: (qmail 4827 invoked by uid 10000); 2 Feb 2010 07:35:40 -0000 Date: Tue, 2 Feb 2010 07:35:40 +0000 From: "Robin H. Johnson" To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] GLEP58 - MetaManifest Message-ID: <20100202073539.GA9387@orbis-terrarum.net> References: <7c612fc61002012227i5a4b2cf5ha08f5a0c87f70ce0@mail.gmail.com> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HG+GLK89HZ1zG0kk" Content-Disposition: inline In-Reply-To: <7c612fc61002012227i5a4b2cf5ha08f5a0c87f70ce0@mail.gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) X-Archives-Salt: 75b01cf8-c196-482a-965d-c929224ffb56 X-Archives-Hash: e55ccdc31273afb01baf156c682d1cfe --HG+GLK89HZ1zG0kk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 01, 2010 at 11:27:01PM -0700, Denis Dupeyron wrote: > You'll find below an email from solar to Robin about MetaManifest. I'm > adding it to this thread (with solar's authorization) as it seems > pertinent. >=20 > Denis. >=20 > On Thu, Jan 21, 2010 at 6:51 PM, Ned Ludd wrote: > > Robin, > > > > I recall you wanted me to mail you what we talked about last nite in > > #gentoo-portage and I'll CC: the council so they have an idea what to > > maybe expect. > > > > So in our talking last night we discussed the fact that if the Manifest > > format has to change why not just get rid of it all together, and save > > some serious in tree space with the new MetaManifest's taking over all > > together. This would include MetaManifest's at the 2-level. > > You said the MetaManifest would need about 4 fields in them to describe > > the distfiles etc. Devs would still push normal Manifest's to the cvs > > tree so DIST can be obtained by the backend infra scripts. But those > > Manifest's could be dropped from the mirroring. if [ -e CVS ] then > > portage would need to use the existing Manifest's First, I'd like to clarify one things for all other readers, as it isn't clear for anybody else just reading this email. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Solar's proposal does the following: 1. Tree in CVS/VCS: - drop ALL Manifest2 lines _EXCEPT_ DIST. 2. Tree available via rsync: - Manifests at the following locations ONLY: - /MetaManifest - /${CAT}/Manifest - /profiles/Manifest - /eclasses/Manifest - /metadata/cache/${CAT}/Manifest - /metadata/glsa/Manifest - Data from ALL Manifests get moved to one of the above. - MISC/EBUILD etc (non-DIST) lines generated at the same time that the rsync tree is prepared. 3. Net savings of approximately 13000 inodes, as the per package Manifest data is now one level up, saving the inode from the package. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Now, I believe that this above should be possible WITHIN the framework of my proposed MetaManifest changes. I specifically stated in GLEP58: =3D=3D=3D The objective of creating the MetaManifest file(s) is to ensure that every single file in the tree occurs in at least one Manifest. =3D=3D=3D My proposals did not cover removing other Manifest files per solar's suggestion, as I perceived that to be a much larger objective than my goal of actually securing the existing tree distribution. I am entirely open to solar's suggestions, in an additional GLEP, as they will require that Portage support IS fully in place, because old versions WILL fail on a tree without per-package Manifest. > > This method would hands down win my vote. As you know I'm not a fan of > > format changes in general as they can make the Gentoo experience suck, > > but if we are going to change formats. Lets do it right. A potential plan for GLEP58 and solar's changes would be: 1. Council approves GLEP58. 2. Portage support is added, we add MetaManifests everywhere needed (top-level, categories, metadata, eclass etc) in the tree. 3. Old Portage versions still work at this point, because they ignore the other Manifest files. 4. Wait 6-12 months for Portage upgrade cycle. 5. Change the content of the MetaManifests to be per solar's proposal. 6. Drop per-package Manifests from the tree. Thus there is ZERO breakage.=20 A similar timeline is required for ALL of the other GLEPs I have proposed. GLEP59 - Hashes: - Can add new hashes right now. - Some of the old hashes we can remove right now. - Have to keep just one old hash for old Portage to still work. GLEP60 - Filetypes: - Can add new types right now. - Cannot remove ANY types for a full upgrade cycle. GLEP61 - Compression: - (uncofirmed) Cannot add the compressed files in per-package locations unt= il the upgrade cycle is done, as old Portage will complain about their exis= tence. > The only downside I can see in this method is for people like drobbins > who mirror our tree but overlay right on top of it then provide it back > out. In such cases we should provide our backend scripts to the public > so they can re MetaManifest. My MetaManifest generation script is already public. I do agree that we cou= ld do better in documenting and publishing our older rsync generation scripts. --=20 Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robbat2@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 --HG+GLK89HZ1zG0kk Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Robbat2 @ Orbis-Terrarum Networks - The text below is a digital signature. If it doesn't make any sense to you, ignore it. iEYEARECAAYFAktn1coACgkQPpIsIjIzwiyNZQCffhn9tQgLKOKn42z3rImayujP bDoAmwR62IMnYpfO8BBDFGxCHok/ysGe =z0iX -----END PGP SIGNATURE----- --HG+GLK89HZ1zG0kk--