Re: [gentoo-portage-dev] Current portage well designed, but badly used

[Gustavo Barbieri <barbieri@gmail.com>]

On Sat, 27 Nov 2004 17:48:54 -0600, Michael Tindal <urilith@gentoo.org> wrote:
> Gustavo Barbieri wrote:
> 
> 
> 
> >Hello,
> >
> >I'm playing with portage and noticed it's well designed, but there are
> >some mistakes in its usage at the moment. For example:
> >
> >Categories are mixed: there is a net-www/apache and net-www/mod_*
> >(apache modules), but there is a more convenient category www-apache/
> >for them. This is one example, there are more mistakes.  There is any
> >plan to fix them in next portage releases?
> >
> >Some packages use numbering version padded with zero, that's good to
> >list with shell functions, but it's bad because you can't change them
> >to numbers and them back to string. For example:
> >mail-mta/nullmailer-1.00_rc7-r4. If you Convert it to integers, it
> >becomes 1.0 and you can't map back to the ebuild.
> >
> >Portage provides metadata.xml, cool. But it's hardly used :(
> >metadata.xml seems to provide tags for maintainers, changelogs and
> >long description, many (most?) packages don't use them.
> >
> >
> All of this you mentioned is really irrelevant to sys-apps/portage, but
> more relevant to the tree.
> The categories you mentioned are held resposible by individual
> maintainers, not the portage team.
> To see who is responsible for the category/package/ebuild, you use the
> metadata.xml.  Packages dont
> need to use the metadata.xml directly because it is just that,
> metadata.  It is used to provide the information
> you just stated, and it serves its purpose well.

I just think that if metadata.xml was filled with long description and
maintainers emails, it would help people over there. Ok, packages
don't need to use it directly, but tools might want to show to users
more info about something.

Talking about metadata, why does HOMEPAGE and DESCRIPTION are in
ebuilds and not in metadata.xml, IMHO they're not used to build the
package in any way.   Maybe if we move those (always filled)
information to metadata.xml, people would fill other fields there.

Also, you said that this is irrelevant to the portage application, but
to the portage tree. Where can I talk to portage tree maintainers? If
I need to patch the entire portage with metadata.xml and stuff like
that, it will be an huge work, but if portage maintainers ask the
package maintainers to do it for next releases, many people would do
small jobs, easier than small group doing many jobs.


> >The portage library is too heavy, complicated and make things slow.
> >Heavy and complicated I noticed from (trying to) look at the source,
> >slow by usage. For example:
> >
> >time emerge # without parameters
> >real    0m0.614s
> >user    0m0.487s
> >sys     0m0.046s
> >
> >time emerge -pv world # 16 packages to be upgraded
> >real    0m22.664s
> >user    0m12.423s
> >sys     0m1.130s
> >
> >It's too much, look at debian apt, it's fast. And I can't see why
> >portage is slow.
> >Forgive me if I'm wrong, but portage just need to parse
> >/var/lib/portage/world (237 entries in my case), them for each check
> >if there is any other version greater than and if so check for
> >dependencies. Why 22seconds? A hand made take less than 1.
> >
> >
> >
> You can't compare apt to portage, dont even try to go down this route.
> Its like comparing apples to lemons.
> Portage is slow, but it is being fixed.  CVS portage for one is a whole
> lot faster.

I'll look at CVS, but I don't see why portage need to be slow. As you
said, it's being fixed.

and I can't see that difference between portage and apt in the area
portage is slow, ok apt uses a db and don't need to check use flags,
but they're orders of magnitude different. Even lemons and apples are
that different ;)


> >Also, a brief explanation on why I was playing with portage and some
> >requests: I'm coding (for fun, no plan to get in a production state)
> >yet another graphical package manager atop portage with the newbie in
> >mind. But to achieve my goal I need:
> >
> >   - a fast portage. Now I'm doing a module to do this for me (see
> >more above), at least the basics, like get package information,
> >versions, ... and if possible resolve primary dependencies (just to
> >show to user in a tab "Dependencies", hidden by default).
> >
> >   - more meta data, if possible a list of urls to screenshots (most
> >packages have a screenshots section), if the url links to an html,
> >provide a threshold of images size to get, so it connects and
> >downloads every image bigger than it... cached of course.
> >
> >
> This is uncessary and would add extra bloat to the tree, and adds more
> complexity to our dev team.
> If you want something like this your best bet would be to provide a
> patch for its functionality on
> bugs.gentoo.org, but I wouldnt be surprised if it wasnt accepted.

I mean I want this in metadata.xml, not in ebuilds or so... how can
this add complexity to dev team? You mean writing the xml parser, it's
easy and I can send the patches.  Also, I can provide tools that check
urls to see if they still exists, like the homepage and screenshots.


> >   - portage to act as a daemon, queue requests  and fetch packages.
> >If portage could be a daemon with 3 threads: one that download
> >packages, one that compiles and one to manage the other and accept
> >requests; then it could schedule download to maximize download
> >throughput, downloading smaller packages first while respecting
> >dependencies, compile while download and wait until packages are there
> >and the "emerge" command just send commands to it.  It would be handy
> >since compiling times are huge.
> >
> >
> There is a daemonized ebuild.sh (correct me if I'm wrong ferringb) in
> portage CVS, although it
> doesnt use threads, because there is no way to kill processes (wget,
> etc.) spawned from within
> a thread, so youd have stale processes after Ctrl+C'ing portage.

Great!
BTW, with threads I meant the concept of more than one thing running
in parallel, don't need to be posix threads, can be process or even
one process using select()


> >About the fast portage: I know portage is a complex monster and is the
> >heart of gentoo, if it breaks, everything breaks. But how about a
> >python module to be used by other packages that just want to view the
> >portage and its packages. If eventually this module works as expected
> >and have every current portage feature, it could replace the old one.
> >
> >
> Jstubbs is working on an api that will make its way into a later
> revision of portage.  As far as parsing
> ebuilds, they are sourced directly from bash.

There is any explanation/roadmap/design I can look at? Jstubbs reads
this list? What's his goals, how he want to achieve it?

About parsing of ebuilds, what do I need to source before the ebuild
itself? I mean, to get things like "inherit" working.

-- 
Gustavo Sverzut Barbieri
---------------------------------------
Computer Engineer 2001 - UNICAMP
GPSL - Grupo Pro Software Livre
Cell..: +55 (19) 9165 8010
Jabber: gsbarbieri@jabber.org
  ICQ#: 17249123
   GPG: 0xB640E1A2 @ wwwkeys.pgp.net

--
gentoo-portage-dev@gentoo.org mailing list