From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 24657 invoked from network); 28 Nov 2004 17:08:39 +0000 Received: from smtp.gentoo.org (156.56.111.197) by lists.gentoo.org with AES256-SHA encrypted SMTP; 28 Nov 2004 17:08:39 +0000 Received: from lists.gentoo.org ([156.56.111.196] helo=parrot.gentoo.org) by smtp.gentoo.org with esmtp (Exim 4.41) id 1CYSXP-0001pD-3h for arch-gentoo-portage-dev@lists.gentoo.org; Sun, 28 Nov 2004 17:08:39 +0000 Received: (qmail 13890 invoked by uid 89); 28 Nov 2004 17:08:37 +0000 Mailing-List: contact gentoo-portage-dev-help@gentoo.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail Reply-To: gentoo-portage-dev@lists.gentoo.org X-BeenThere: gentoo-portage-dev@gentoo.org Received: (qmail 3376 invoked from network); 28 Nov 2004 17:08:37 +0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=AP+KztIReC0pzDQxjp5MdtKx24EsYWwWMrmce+lMj1LAjyzs9SJuNgJiZODLoY14FkY3ZE7GSi03U660SxrSMYFSKO1OKbnYanf9SbOsP+V/jIs9cPYrg2gLMIZcZBVv1wMaV6R9nHTqefBzEQAnmJxzo/CtiFHjFpVJLm5WqLc= Message-ID: <9ef20ef3041128090844573b74@mail.gmail.com> Date: Sun, 28 Nov 2004 15:08:06 -0200 From: Gustavo Barbieri Reply-To: Gustavo Barbieri To: gentoo-portage-dev@lists.gentoo.org In-Reply-To: <41A91266.5080804@gentoo.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <9ef20ef3041127151046107fb5@mail.gmail.com> <41A91266.5080804@gentoo.org> Subject: Re: [gentoo-portage-dev] Current portage well designed, but badly used X-Archives-Salt: a1482d33-30ed-4081-8cd4-655492eb9a87 X-Archives-Hash: 7812f8ad541e44b18481556adf5102d1 On Sat, 27 Nov 2004 17:48:54 -0600, Michael Tindal wrote: > Gustavo Barbieri wrote: > > > > >Hello, > > > >I'm playing with portage and noticed it's well designed, but there are > >some mistakes in its usage at the moment. For example: > > > >Categories are mixed: there is a net-www/apache and net-www/mod_* > >(apache modules), but there is a more convenient category www-apache/ > >for them. This is one example, there are more mistakes. There is any > >plan to fix them in next portage releases? > > > >Some packages use numbering version padded with zero, that's good to > >list with shell functions, but it's bad because you can't change them > >to numbers and them back to string. For example: > >mail-mta/nullmailer-1.00_rc7-r4. If you Convert it to integers, it > >becomes 1.0 and you can't map back to the ebuild. > > > >Portage provides metadata.xml, cool. But it's hardly used :( > >metadata.xml seems to provide tags for maintainers, changelogs and > >long description, many (most?) packages don't use them. > > > > > All of this you mentioned is really irrelevant to sys-apps/portage, but > more relevant to the tree. > The categories you mentioned are held resposible by individual > maintainers, not the portage team. > To see who is responsible for the category/package/ebuild, you use the > metadata.xml. Packages dont > need to use the metadata.xml directly because it is just that, > metadata. It is used to provide the information > you just stated, and it serves its purpose well. I just think that if metadata.xml was filled with long description and maintainers emails, it would help people over there. Ok, packages don't need to use it directly, but tools might want to show to users more info about something. Talking about metadata, why does HOMEPAGE and DESCRIPTION are in ebuilds and not in metadata.xml, IMHO they're not used to build the package in any way. Maybe if we move those (always filled) information to metadata.xml, people would fill other fields there. Also, you said that this is irrelevant to the portage application, but to the portage tree. Where can I talk to portage tree maintainers? If I need to patch the entire portage with metadata.xml and stuff like that, it will be an huge work, but if portage maintainers ask the package maintainers to do it for next releases, many people would do small jobs, easier than small group doing many jobs. > >The portage library is too heavy, complicated and make things slow. > >Heavy and complicated I noticed from (trying to) look at the source, > >slow by usage. For example: > > > >time emerge # without parameters > >real 0m0.614s > >user 0m0.487s > >sys 0m0.046s > > > >time emerge -pv world # 16 packages to be upgraded > >real 0m22.664s > >user 0m12.423s > >sys 0m1.130s > > > >It's too much, look at debian apt, it's fast. And I can't see why > >portage is slow. > >Forgive me if I'm wrong, but portage just need to parse > >/var/lib/portage/world (237 entries in my case), them for each check > >if there is any other version greater than and if so check for > >dependencies. Why 22seconds? A hand made take less than 1. > > > > > > > You can't compare apt to portage, dont even try to go down this route. > Its like comparing apples to lemons. > Portage is slow, but it is being fixed. CVS portage for one is a whole > lot faster. I'll look at CVS, but I don't see why portage need to be slow. As you said, it's being fixed. and I can't see that difference between portage and apt in the area portage is slow, ok apt uses a db and don't need to check use flags, but they're orders of magnitude different. Even lemons and apples are that different ;) > >Also, a brief explanation on why I was playing with portage and some > >requests: I'm coding (for fun, no plan to get in a production state) > >yet another graphical package manager atop portage with the newbie in > >mind. But to achieve my goal I need: > > > > - a fast portage. Now I'm doing a module to do this for me (see > >more above), at least the basics, like get package information, > >versions, ... and if possible resolve primary dependencies (just to > >show to user in a tab "Dependencies", hidden by default). > > > > - more meta data, if possible a list of urls to screenshots (most > >packages have a screenshots section), if the url links to an html, > >provide a threshold of images size to get, so it connects and > >downloads every image bigger than it... cached of course. > > > > > This is uncessary and would add extra bloat to the tree, and adds more > complexity to our dev team. > If you want something like this your best bet would be to provide a > patch for its functionality on > bugs.gentoo.org, but I wouldnt be surprised if it wasnt accepted. I mean I want this in metadata.xml, not in ebuilds or so... how can this add complexity to dev team? You mean writing the xml parser, it's easy and I can send the patches. Also, I can provide tools that check urls to see if they still exists, like the homepage and screenshots. > > - portage to act as a daemon, queue requests and fetch packages. > >If portage could be a daemon with 3 threads: one that download > >packages, one that compiles and one to manage the other and accept > >requests; then it could schedule download to maximize download > >throughput, downloading smaller packages first while respecting > >dependencies, compile while download and wait until packages are there > >and the "emerge" command just send commands to it. It would be handy > >since compiling times are huge. > > > > > There is a daemonized ebuild.sh (correct me if I'm wrong ferringb) in > portage CVS, although it > doesnt use threads, because there is no way to kill processes (wget, > etc.) spawned from within > a thread, so youd have stale processes after Ctrl+C'ing portage. Great! BTW, with threads I meant the concept of more than one thing running in parallel, don't need to be posix threads, can be process or even one process using select() > >About the fast portage: I know portage is a complex monster and is the > >heart of gentoo, if it breaks, everything breaks. But how about a > >python module to be used by other packages that just want to view the > >portage and its packages. If eventually this module works as expected > >and have every current portage feature, it could replace the old one. > > > > > Jstubbs is working on an api that will make its way into a later > revision of portage. As far as parsing > ebuilds, they are sourced directly from bash. There is any explanation/roadmap/design I can look at? Jstubbs reads this list? What's his goals, how he want to achieve it? About parsing of ebuilds, what do I need to source before the ebuild itself? I mean, to get things like "inherit" working. -- Gustavo Sverzut Barbieri --------------------------------------- Computer Engineer 2001 - UNICAMP GPSL - Grupo Pro Software Livre Cell..: +55 (19) 9165 8010 Jabber: gsbarbieri@jabber.org ICQ#: 17249123 GPG: 0xB640E1A2 @ wwwkeys.pgp.net -- gentoo-portage-dev@gentoo.org mailing list