Re: [gentoo-dev] Proposal of change in emerge

Re: [gentoo-dev] Proposal of change in emerge

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

On Sun, 26 Sep 2004 16:11:45 +0200 Abraham Marin Perez
<abraham@alumni.uv.es> wrote:
|     This example is quite simple, but that's the idea: find out which
| packages are independent and which should wait for some others to be
| merged, separate the whole process in parts and run in parallel
| independent tasks.

http://bugs.gentoo.org/show_bug.cgi?id=1661

-- 
Ciaran McCreesh : Gentoo Developer (Sparc, MIPS, Vim, Fluxbox)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Proposal of change in emerge

[Abraham Marin Perez]

Hi everyone,

    I've been thinking about a new way to make emerge works, it's just
an idea and I haven't thought about the code itself, and actually I have
never used python nor hacked portage code, but I thought it'd be good to
tell it. Here it goes:

    Let's say we want to install a package X and that it needs packages
Y and Z to be emerged. However, Y and Z don't depend in each other, so
we can do some kind of parallel merging to speed up the whole process:
we download Y and then we start Y's compilation, but, since we don't use
the network during the compilation, we can safely download Z meanwhile.
When Y's compilation and Z's downloading are over, we can compile Z and
download X while Y is being merged.

    This example is quite simple, but that's the idea: find out which
packages are independent and which should wait for some others to be
merged, separate the whole process in parts and run in parallel
independent tasks. Portage has now a main core who calculates
dependences and "independences" and uses three agents: the downloader,
the builder and the installer. If we have something to download we check
whether the downloader is free, if so, we send it the task. If the
compiler is free and we have an already downloaded file that doesn't
need any other package to be merged, we send it the task. If the
installer is free and we have a compiled packaged, we use the installer
to merge it.

    On the other hand, if you try to emerge more than one package at the
same time and they happen to share dependences they may try to emerge
them simultaneously, causing several problems. Hence, we could make
portage works like a server that merges simultaneous install petitions
and check for shared dependences. Of course, it should keep the original
package to know what should be added to world file and which tasks
should be stopped if the main job is cancelled.

    I hope I didn't state the obvious, as I said, I haven't taken a look
to portage code but, according the way it works, this new way could make
it faster and avoid some problems. What do you think about it?

Regards,
Abraham


************************************************************************
Abraham Marin Perez
Studying Computer Science at the University of Valencia
E-mail: abraham@alumni.uv.es
Home page: http://mural.uv.es/abraham

"Sorry, this virus requires Microsoft Windows", Blaster.C
************************************************************************


--
gentoo-dev@gentoo.org mailing list