From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 18812 invoked from network); 28 Nov 2004 19:37:56 +0000 Received: from smtp.gentoo.org (156.56.111.197) by lists.gentoo.org with AES256-SHA encrypted SMTP; 28 Nov 2004 19:37:56 +0000 Received: from lists.gentoo.org ([156.56.111.196] helo=parrot.gentoo.org) by smtp.gentoo.org with esmtp (Exim 4.41) id 1CYUrs-00039K-1v for arch-gentoo-portage-dev@lists.gentoo.org; Sun, 28 Nov 2004 19:37:56 +0000 Received: (qmail 7529 invoked by uid 89); 28 Nov 2004 19:37:54 +0000 Mailing-List: contact gentoo-portage-dev-help@gentoo.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail Reply-To: gentoo-portage-dev@lists.gentoo.org X-BeenThere: gentoo-portage-dev@gentoo.org Received: (qmail 26366 invoked from network); 28 Nov 2004 19:37:54 +0000 From: Paul de Vrieze To: gentoo-portage-dev@lists.gentoo.org Date: Sun, 28 Nov 2004 20:37:58 +0100 User-Agent: KMail/1.7.1 References: <9ef20ef3041127151046107fb5@mail.gmail.com> In-Reply-To: <9ef20ef3041127151046107fb5@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1591565.Jk4OasRPDB"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200411282037.58501.pauldv@gentoo.org> Subject: Re: [gentoo-portage-dev] Current portage well designed, but badly used X-Archives-Salt: 66a7a322-251f-41be-b26c-d03e9c7ab983 X-Archives-Hash: 185589d30c1564e4d93093d92de4955c --nextPart1591565.Jk4OasRPDB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sunday 28 November 2004 00:10, Gustavo Barbieri wrote: > Hello, > > I'm playing with portage and noticed it's well designed, but there are > some mistakes in its usage at the moment. For example: > > Categories are mixed: there is a net-www/apache and net-www/mod_* > (apache modules), but there is a more convenient category www-apache/ > for them. This is one example, there are more mistakes. There is any > plan to fix them in next portage releases? > > Some packages use numbering version padded with zero, that's good to > list with shell functions, but it's bad because you can't change them > to numbers and them back to string. For example: > mail-mta/nullmailer-1.00_rc7-r4. If you Convert it to integers, it > becomes 1.0 and you can't map back to the ebuild. > > Portage provides metadata.xml, cool. But it's hardly used :( > metadata.xml seems to provide tags for maintainers, changelogs and > long description, many (most?) packages don't use them. > > The portage library is too heavy, complicated and make things slow. > Heavy and complicated I noticed from (trying to) look at the source, > slow by usage. For example: > > time emerge # without parameters > real 0m0.614s > user 0m0.487s > sys 0m0.046s > > time emerge -pv world # 16 packages to be upgraded > real 0m22.664s > user 0m12.423s > sys 0m1.130s > > It's too much, look at debian apt, it's fast. And I can't see why > portage is slow. > Forgive me if I'm wrong, but portage just need to parse > /var/lib/portage/world (237 entries in my case), them for each check > if there is any other version greater than and if so check for > dependencies. Why 22seconds? A hand made take less than 1. > > > Also, a brief explanation on why I was playing with portage and some > requests: I'm coding (for fun, no plan to get in a production state) > yet another graphical package manager atop portage with the newbie in > mind. But to achieve my goal I need: > > - a fast portage. Now I'm doing a module to do this for me (see > more above), at least the basics, like get package information, > versions, ... and if possible resolve primary dependencies (just to > show to user in a tab "Dependencies", hidden by default). > > - more meta data, if possible a list of urls to screenshots (most > packages have a screenshots section), if the url links to an html, > provide a threshold of images size to get, so it connects and > downloads every image bigger than it... cached of course. > > - portage to act as a daemon, queue requests and fetch packages. > If portage could be a daemon with 3 threads: one that download > packages, one that compiles and one to manage the other and accept > requests; then it could schedule download to maximize download > throughput, downloading smaller packages first while respecting > dependencies, compile while download and wait until packages are there > and the "emerge" command just send commands to it. It would be handy > since compiling times are huge. > > > About the fast portage: I know portage is a complex monster and is the > heart of gentoo, if it breaks, everything breaks. But how about a > python module to be used by other packages that just want to view the > portage and its packages. If eventually this module works as expected > and have every current portage feature, it could replace the old one. > I started to code my own "fast portage", but some things are picky > to do, and I want to know how you do that: how do you parse ebuilds to > get USE, DESCRIPTION, SLOT, DEPEND, ... ? > If you want to know why my implementation is fast: I use lazy > evaluation as far as possible. For example, I load every package, but > the attributes to available versions, installed versions, the status, > are just calculated on deman, I use python property() and > setters/getters for that. Since hardly you'll use every attribute from > everythin, it loads much faster. > I have preliminar code here: > http://ltc08.ic.unicamp.br/~gustavo/packagemanager.tar.bz2, but some > modifications I did were lost in a power outtage + xfs... I just have > the .pyc, if someone knows how to get the .py back... Well, as said, portage does not parse by itself, but uses bash. This is not= =20 really fast. The biggest issue however is the absense of lazy evaluation.=20 I've been looking at it too, and even have a c++ based parser that can be=20 accessed as a python module, but it's undocumented and has issues as it is= =20 not a full bash replacement, and ebuilds expect bash. Paul =2D-=20 Paul de Vrieze Gentoo Developer Mail: pauldv@gentoo.org Homepage: http://www.devrieze.net --nextPart1591565.Jk4OasRPDB Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (GNU/Linux) iD8DBQBBqikWbKx5DBjWFdsRAq+VAKCFQsyip/5A681OW1lV2EG0lod84ACdFkRR uIvOJiBZntShynh2u9gglpg= =c+MY -----END PGP SIGNATURE----- --nextPart1591565.Jk4OasRPDB--