Re: [gentoo-dev] has_version etc parallelisability

[Brian Harring <ferringb@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2172 bytes --]

On Mon, Dec 31, 2007 at 02:28:44PM +0000, Ciaran McCreesh wrote:
> On Sun, 30 Dec 2007 20:11:16 -0800
> "Alec Warner" <antarus@gentoo.org> wrote:
> > On 12/30/07, Ciaran McCreesh <ciaran.mccreesh@blueyonder.co.uk> wrote:
> > > Is it legal for ebuilds to call has_version and friends in
> > > parallel? Is it legal for ebuilds to call has_version and friends
> > > after the ebuild process has terminated? Discuss.
> > 
> > If the pm implements read/write locking on the underlying datastore
> > (which it should probably have regardless of this request) then I
> > don't see a problem in parallel has_version calls.
> 
> Actually, it's the communication channel that's the issue... If, for
> example, has_version is implemented in terms of a request on a pipe
> rather than execing a new package manager, we get into messy bash
> locking territory...
> 
> > I don't get your second example..do you mean the ebuild is running
> > has_version in the background and then terminating?
> 
> Yeah. Again, consider the pipe example. If the package manager closes
> off the pipe when it thinks the ebuild's done, calling has_version will
> get the backgrounded process SIGPIPEd.

Depends on the implementation; for pkgcore, if that comm pipe is 
dead, the ebuild env *should* be dead, or dieing.  Background'ing 
processes from that env isn't valid imo, either.

If you're refering to an ebuild that parallelizes itself while 
executing, iow, parallelization w/in the ebuild env/phase execution, 
I'd look more at being able to batch commands instead of trying to run 
them in parallel.  Reasoning follows-

1) if doing an exec approach to service the request, this means 
reparsing of involved files for each request- inefficient, potentially 
horribly so on crappy hardware/setups.
2) screws up the pipe approach, should folks take it for control/env 
introspection gains.

Summarizing, executing has_version (and friends) 
concurrently has it's own issues performance wise, and implementation 
wise; growing batch functionality into portageq however avoids those 
issues, and would be faster- thus the route I'd advocate.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]