[gentoo-portage-dev] speedup emerge?

[gentoo-portage-dev] speedup emerge?

[myfirstname mylastname]

[-- Attachment #1: Type: text/plain, Size: 1334 bytes --]

Hi there, 

I installed GENTOO yeserday and I must say:

I LIKE IT.

Just some observations though:

when emerging my way from stage 1 to three, I noticed that the emerge
does a lot of stuff serially. It first tries to find out a lot of stuff, then downloads the package, then compiles and then checks. These stages have to wait for
each other to complete. 

Would it be an idea to run these tasks parallelly like:

emerge.findout|emerge.download|emerge.build|emerge.check_and_log;

the processes could pass an xml job describing structure like :
job
  package
    file
      downloaded_ok
      compiled_ok
      installed_ok
      various messages and flags
    /file
  /packaeg
/job. 

this way downloading, checking and building are done in parrallel.
Extra advantage: the logging process has all relevant status
info at hand and can draw intelligent conclusions as to what went
wrong. 

It's just a thought; looking at the emerge script, I couldn's find
obvious points for cutting it in four. I noticed that messages are
writen to stderr, so piping stdouts to stdins should be no problem. 

Any thoughts on it? 

cheers, 

cobus
-- 
_______________________________________________
Find what you are looking for with the Lycos Yellow Pages
http://r.lycos.com/r/yp_emailfooter/http://yellowpages.lycos.com/default.asp?SRC=lycos10


[-- Attachment #2: Type: message/rfc822, Size: 4205 bytes --]

From: MAILER-DAEMON@parrot.gentoo.org
To: cobus@lycos.com
Subject: failure notice
Date: 15 Aug 2004 10:14:54 +0000
Message-ID: <20040815101458.66B6837032@spf7-14.us4.outblaze.com>

Hi. This is the qmail-send program at parrot.gentoo.org.
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.

<gentoo-portage-dev@lists.gentoo.org>:
Sorry, only subscribers may post. If you are a subscriber, please forward this message to gentoo-portage-dev-owner@gentoo.org to get your new address included (#5.7.2)

--- Below this line is a copy of the message.

Return-Path: <cobus@lycos.com>
Received: (qmail 28430 invoked from network); 15 Aug 2004 10:14:54 +0000
Received: from smtp.gentoo.org (156.56.111.197)
  by lists.gentoo.org with AES256-SHA encrypted SMTP; 15 Aug 2004 10:14:54 +0000
Received: from webmail-outgoing.us4.outblaze.com ([205.158.62.67])
	by smtp.gentoo.org with esmtp (Exim 4.34)
	id 1BwI2P-0006oQ-Vf
	for gentoo-portage-dev@lists.gentoo.org; Sun, 15 Aug 2004 10:14:54 +0000
Received: from wfilter.us4.outblaze.com (wfilter.us4.outblaze.com [205.158.62.180])
	by webmail-outgoing.us4.outblaze.com (Postfix) with QMQP id 9AA8A1800796
	for <gentoo-portage-dev@gentoo.org>; Sun, 15 Aug 2004 10:14:53 +0000 (GMT)
X-OB-Received: from unknown (208.36.123.34)
  by wfilter.us4.outblaze.com; 15 Aug 2004 10:14:53 -0000
Received: by ws7-5.us4.outblaze.com (Postfix, from userid 1001)
	id 88A75C610C; Sun, 15 Aug 2004 10:14:53 +0000 (GMT)
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-Mailer: MIME-tools 5.41 (Entity 5.404)
Received: from [80.127.44.35] by ws7-5.us4.outblaze.com with http for
    cobus@lycos.com; Sun, 15 Aug 2004 05:14:53 -0500
From: "myfirstname mylastname" <cobus@lycos.com>
To: gentoo-portage-dev@lists.gentoo.org
Date: Sun, 15 Aug 2004 05:14:53 -0500
Subject: emerge speedup?
X-Originating-Ip: 80.127.44.35
X-Originating-Server: ws7-5.us4.outblaze.com
Message-Id: <20040815101453.88A75C610C@ws7-5.us4.outblaze.com>

Hi there, 

I installed GENTOO yeserday and I must say:

I LIKE IT.

Just some observations though:

when emerging my way from stage 1 to three, I noticed that the emerge script first tries to find out a lot of stuff, then downloads the package, then compiles and then checks. These stages have to wait for each other to complete. 

Would it be an idea to run these tasks parallelly like:

emerge.findout|emerge.download|emerge.build|emerge.check_and_log;

the processes could pass an xml struct like :
job
  package
    file
      downloaded_ok
      compiled_ok
      installed_ok
      etc
    /file
  /packaeg
/job. 

this way downloading, checking and building are done in parrallel. Extra advantage is, that the logging process has all relevant status info at hand and can draw intelligent conclusions as to what went wrong. 

It's just a thought; looking at the emerge script, I couldn's find obvious points for cutting it in four. I noticed that messages are writen to stderr, so piping stdout to stdin would be no problem. 

Any thoughts on it? 

cheers, 

cobus



-- 
_______________________________________________
Find what you are looking for with the Lycos Yellow Pages
http://r.lycos.com/r/yp_emailfooter/http://yellowpages.lycos.com/default.asp?SRC=lycos10



[-- Attachment #3: Type: text/plain, Size: 45 bytes --]

--
gentoo-portage-dev@gentoo.org mailing list