From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 12225 invoked from network); 12 Jun 2004 00:52:50 +0000 Received: from smtp.gentoo.org (156.56.111.197) by lists.gentoo.org with AES256-SHA encrypted SMTP; 12 Jun 2004 00:52:50 +0000 Received: from lists.gentoo.org ([156.56.111.196] helo=parrot.gentoo.org) by smtp.gentoo.org with esmtp (Exim 4.34) id 1BYwlM-0004ty-Sg for arch-gentoo-portage-dev@lists.gentoo.org; Sat, 12 Jun 2004 00:52:49 +0000 Received: (qmail 903 invoked by uid 89); 12 Jun 2004 00:52:47 +0000 Mailing-List: contact gentoo-portage-dev-help@gentoo.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail Reply-To: gentoo-portage-dev@lists.gentoo.org X-BeenThere: gentoo-portage-dev@gentoo.org Received: (qmail 25765 invoked from network); 12 Jun 2004 00:52:47 +0000 From: John Nilsson To: lafou@wanadoo.fr Cc: gentoo-portage-dev@lists.gentoo.org In-Reply-To: <40CA20FD.8040907@wanadoo.fr> References: <40CA20FD.8040907@wanadoo.fr> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-113zrSOG/SooRis+P76P" Message-Id: <1087001708.28752.28.camel@newkid.milsson.nu> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Sat, 12 Jun 2004 02:55:09 +0200 Subject: Re: [gentoo-portage-dev] portage doc & cache X-Archives-Salt: 911c7112-2772-4b6d-be5a-4b4dfaf685a6 X-Archives-Hash: e3eb5ba0dd01b7857de93303f4f8d99d --=-113zrSOG/SooRis+P76P Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Fri, 2004-06-11 at 23:15, Philippe Lafoucri=E8re wrote: > Hi all >=20 > I'm wondering where I can find some doc about portage, and especially=20 > the cache part. I use debian at work, and get so sad when doing an=20 > emerge rsync at home (10 or 15 minutes to rsync, and almost 5 minutes to=20 > calculate world dependancies) on a bi-athlon 1.8GHz. >=20 > I just wanted to check that the cache system is really a cache :) >=20 > maybe ebuilds should be "compiled" into seriaziled objects or something=20 > similar. Only an emerge rsync (or something called "emerge update")=20 > would fetch new ebuilds, and compile new ones (we don't care anymore of=20 > hand modified files after sync). Mabey rfc2616 can be of some nice inspiration about caching. As always, I refer to Roy T. Fielding's "Architectural Styles and the Design of Network-based Software Architectures" as a must read. >=20 >=20 > On the other hand, I was thinkink of the "rsync way" to synchronise the=20 > portage tree. Why don't we use a portage tree ID (incremented each time=20 > an ebuild is commited to CVS). For exemple, I have the tree number=20 > 12512. Current portage tree ID is 12514. I just need too fresh ebuilds. You have just described subversion... Anyway you might want to experiment with cvsup vs. rsync. > Moreover, do I REALLY need all ebuilds content ???? Why can't portage=20 > just look at some headers (with arch, desc, etc.) and left the install=20 > part on other files. Rsync local tree would be lighter, and faster to=20 > sync. When you need to emerge some program, emerge would download the=20 > _real_ ebuild file, the digests, the distfiles, etc. Do some experiments and benchmarking. What if only the cache was downloaded at sync? Say you have a sql backend for portage, just sync that and download the ebuilds on demand. > The headers can also be contained in one large tar.gz file, to reduce=20 > bandwith again (I know, like debian). >=20 > I know this stuff might let you think I'm crazy to rebuild all portage,=20 > but here are some ideas to dig :D Personally I like crazy =3D) I think you should have a better understanding of the current system before you criticise it though.=20 rsync is a file transfer program for Unix systems. rsync uses the "rsync algorithm" which provides a very fast method for bringing remote files into sync. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand. rsync allready does it's best to only download new information. -John --=-113zrSOG/SooRis+P76P Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQBAylRs0GAlrvwEkG4RAgdPAJ4lw4SjRZv57KiLatryFnE6pkQeWQCeNowg f1eLr/zWteQrsFc2qxv+qiI= =FDEl -----END PGP SIGNATURE----- --=-113zrSOG/SooRis+P76P--